Angry Protocols

“That’s an interesting problem. I think it’s time for . . . an ANGRY PROTOCOL!”

“Angry protocol?”

“You know, like a normal protocol, but it’s got knuckle-busters and a ripped leather jacket and biker buddies, and it doesn’t have error conditions, it TERMINATES. Port numbers have to be pried from its cold, dead fingers, and if you don’t have a couple dozen cores and eight gig of RAM, you’re just wasting its time. And you don’t want to waste its time, right?”

“Okay. Well, we just want to transfer some files over –”

“I think it’s definitely time for . . . a MEAN BASTARD SON OF A BITCH PROTOCOL STANDARD!”

“You’re at it again.”

“No really. Most standards are full of weasel words, like SHALL NOT (which means MAYBE) and MUST (which also means MAYBE) and REQUIRED TO (which means MAYBE but you’re supposed to feel a little guilty) and MAY (which means DO SOMETHING FUCKED UP AND RANDOM).”

“Everybody knows /that/ stuff. It’s standards 101. But all we –”

“So a Mean Bastard Son-of-a-Bitch Standard is one that you don’t DARE ignore. It’s written with language like MOTHERFUCKING MANDATORY and WE WILL KILL YOU TO DEATH IF YOU DO THIS ANY OTHER WAY, YOU MISERABLE EXCUSE FOR AN ENGINEER and THE LAST GUY WHO TRIED THAT FOUND HIMSELF WIRED IN SERIES IN THE BATTERY ROOM AT THE SERVER FARM.”

“Well, most of the people who implement standards are pansies and a little conductive, sure.”

“Right! Out with the old, safe junk. No X- headers for us. We go right to XXX- and to hell with the lilly-livered still-living-at-their-parents-house weaklings who faint at the sight of a little skin. We’re going to have massive, TOWERING buffers with no slow start, just WHAM, BAM and those bits had better be across or we’re going to start taking files hostage and erasing them one by one until the bandwidth comes up. Flow control is for COWARDS! We don’t negotiate, we TAKE and we CONSUME and we BURN SHIT UP.”

“I think . . . we’ll just mail some thumb drives to our customers.”

“No UTF-256 encoded headers?”

“Sorry.”

“No 10240 bit certificates? No dedicated fiber to the backup servers in the Rockies?”

“No.”

“Well, make sure you get the kind of thumb drive that’s armored and guaranteed waterproof to three hundred feet.”

“Go away.”

Clownspace

What does a buffoon do when his project is utterly out of code space, running late, and rather horribly extremely buggy? Why, he adds more features, of course!

Customer: “We want a log of the temperature readings for the last week, in degrees Fahrenheit _and_ centigrade.”

Contractor-Buffoon Figby: “Okay!”

Customer: “A log of the last week’s current draw would be neat to have, too.”

Contractor Figby: “Yup, gotcha covered!”

Customer: “We’d like to have the following eighteen things done on the wire protocol so that our techs don’t have to upgrade their versions of LabView, because that costs money and you always say ‘Yes’ to us.”

Contractor Figby: “Yes! Sure! Whatever you want!”

The problem was that Figby the Contractor couldn’t program his way out of a paper bag. His code didn’t work (it was riddled with bugs he was in denial about), and he was out of room in his part’s code space by a factor of three (and that was just for the boot loader of the code he hadn’t written yet).

Figby didn’t know how to say “No” . . . so we said it for him.

We tried being gentle (“Why don’t you do it this way? You’d save a lot of space and it would be faster and more reliable”). We talked and filled up some whiteboards with reasonable designs.  No go; he was addicted to making his management happy by giving them — well, more accurately by PROMISING them — whatever they asked for.

We tried _not_ being gentle (“If you continue the way you’re going, you’re going to fail and get a lot of people upset at you”). This entered his ears and went into his brain and somehow got processed as “I need to add more features so I don’t fail.”

Figby had been hired into a group notoriously bad at hiring software people. We tried going through that other group’s management (“He’s going to fail”) and our own management (“That contractor and that management over there are all going to fail”). Nothing worked. We were just the usual high-strung software guys with our shorts in a twist about nothing important. If we’d hired a brass band and done a parade in the parking lot with a big float and Figby in it with garlands holding a banner saying “My code is too big and will be late and it won’t work anyway” we’d have been ignored just as all the other software engineers, always Cassandras, have been ignored from the very first big disaster, probably some spec work for Hammurabi involving prayers for rain that didn’t deliver on time.

Status: We had tried the talking thing, and failed. The buffoon was going to ruin our project, make our lives miserable, and pile the blame on us. It was time to get out the knives.

—-

A little technical background: The piece of code in question was for a small microcontroller. Not the smallest controller we could buy, but darn close: It could hold 4K of code, and it had comic relief quantities of RAM and some EEPROM. The part didn’t need to do very much, but what it had to do was critical; if its code didn’t work, the whole product would be a brick because this little fella was responsible for turning the unit’s power on and off at the right time.

To be safely field-updatable, two copies of the code had to fit in that 4K of flash so that two fully functional images would always be present. Figby the Contractor’s boot loader was 3K and thus well over budget, and not only wasn’t he done, he was still adding and promising new features while ignoring his backlog of bugs.  “Oh, don’t worry,” he said, “The code will just have to go on a diet.”  We were pretty sure that no amount of dieting was feasible.

We were ignored in meetings. We had trouble getting any of the hardware, much of which was reserved for testing (it was unclear what they were testing, with so little of the code working). The train wreck slid closer and closer to us, with Figby yanking incompetently at the controls and apparently still promising that things would be working fine. After a little dieting and bug fixing, of course.

By late November I’d had enough, and I snapped.

—-

How I Spent My Thanksgiving

On the Wednesday before Thanksgiving I went home in a fuggy depression. The next month wasn’t going to be fun. The months after that were not going to be fun, either, since we were the poor bastards who were going to have to make the thing work after Figby finished his contract and skated off to his next disaster.

No hardware? Why should a little thing like that stop a software guy?

Thursday morning I downloaded the specs for the microcontroller from the manufacturer’s public web site. Then my family drove to my in-law’s house, but I’m afraid I wasn’t very social. When we got there I found a comfy chair and wrote some perl to extract the opcode tables and assembler syntax from the chip’s documentation. I started hacking.  By early evening, after a Thanksgiving dinner that couldn’t beat, I had an emulator for the chip’s CPU running some test instructions.

The next day I got a bunch of the chip’s hardware registers emulated, as well as emulated RAM, EEPROM and Flash.

The toolchain vendor had license-protected the C compiler so I couldn’t use it on my laptop at home, but the assembler ran just fine (otherwise I would have written one; probably would have taken three or four hours with the tables I’d already made). So on Saturday I wrote the boot code and got the emulator to run the code and print “Hello, world!” out a simulated serial port.

On Monday I spent an hour or so getting the code to run on the real hardware (I spent half an hour of this sneaking over to the hardware group to “permanently borrow” some hardware and a JTAG probe from one of the labs they’d forgotten I’d been given access to on a prior project). The code didn’t work the first time, but after I set up the baud rate correctly, the second time it ran it printed out “Hello, world!”.  Carriage return. Motherfucking linefeed.  Yeah.

Then I told my boss what I’d done.

“Don’t work on this,” He said. “It’s going to be a big problem and cause a lot of friction.” He winked hard. I think he gave himself an injury making that wink.

“Yeah,” I said, [WINK], “But it’s going to save the project.”

“I know, but it’s going to cause trouble.”

So on Tuesday he called me to his office and said, “I talked to some people. Go go go go, finish that up. Just don’t say anything. We’ll work it out.”

And I wrote the rest of the code in C and assembly in a week or two, redoing what had taken the contractor four or five months.

—-

My cow-orker Nate was jealous. “You’re having too much fun,” he accused.

“I am.” In fact, I did feel bad about it. I was having a grand time writing low-level code in C and assembly, and Nate was working on something in C++ that was definitely not enjoyable.

I checked in some code one evening. The next morning Nate came up to me and said, “Sorry, I couldn’t stay away. I fixed a bug in one of your routines last night.”

“Okay,” I said.

“Then I ran out of space and had to re-code some other routines in assembly to get enough bytes back.”

“Sure, feel free.”

“Holy crap, this is fun.”

And with that, we were a team. It went back and forth like that for a while. I would fix a bug, or Nate would, and we’d need to scavenge bytes by recoding the somewhat fat code that the C compiler emitted, and we’d often wind up with more bytes free than we’d started with. I showed Nate how to multiply by ten, a hundred, and one thousand using a little inline recursion trick, and he called me a sick fuck. Nate rewrote the little build system I’d done and saved even more bytes. It became a contest.

We wound up with six bytes to spare, and Nate said he felt like he’d been on a two year course of anti-depressants.

A few months later we had a last bug fix and had to scavenge more space.  We wound up with six bytes free again after the changes, and those bytes were pretty tough to find.

A manager came up to me a few days later and asked for a feature.

“Nope,” I said, “We’re out of bytes.”

“But it’s a bug.” (I guess the lack of that feature was a bug to him).

“We fixed the last bug on Tuesday. We don’t have space to fix any more. We’re done.”

And that was that. It was nice to be done.

—-

For the entire project, Nate and I never used a debugger; we just did a little strategic output on the serial port. We could have used a JTAG debugger, but after a while it was just easier not to bother.

—-

What happened to Figby?

About a month after we got our version of the firmware working, the other group told him he wasn’t needed. That’s the way it is with contractors. ->Poof!<-

His code, utterly useless, was simply deleted from the repository. Although we would probably have preferred to nail it against the wall, as a warning to future contractors.

“What’s that hard drive doing up there, with the spike through it?”

“Let me tell you a story….”

—-

And the morals of the story are:

1. Crappy and entrenched workers are a fact of life in most big organizations. They can’t necessarily be counted on to spontaneously self-immolate, so you have to be prepared to end-run them.

2. When you do that, get management buy-in at the right time. Like, when you have things largely working and you’ve dealt with (or at least can enumerate) most of the risk.

3. Remember to appear humble and apologetic as you crush their spirits.

4. When you see a software guy walking down the hall holding an oscilloscope, smile.

PHP versus PCP

One of these makes your eyes roll back in their sockets, makes you violent, and causes you to shiver and shake and shout and want to kill things. It makes your eyes bleed. Sharp objects appear inviting and cuddlesome. The sun becomes your enemy. Spray-on hair seems like a plausible thing.

The other is a horrible drug. [Yes, a tired joke, I know…]

I wrote a non-trivial amount of PHP a little while ago, and I simply cannot get clean enough.