This is a war story from lecture about two optimizations in early Windows that were each fine alone and broke the system together.

The two optimizations

RtlZeroMemory zeroes a buffer, and the way to make it fast is to use the widest register available so each instruction moves more bytes. Early Windows picked the biggest registers on the machine, the floating point registers. Copying memory got the same treatment in RtlCopyMemory.

Separately, Dave Cutler optimized interrupt handling to save only the registers a device driver would need. Nobody expected a device driver to do floating point arithmetic, so the floating point registers went unsaved.

The bug

Everyone on the Windows team ran nightly stress tests on each new build, including nightly bug checking of parts of the file system on multiple test machines. A showstopping bug that was blue screening many of the test systems got assigned to Gary. He manually checked whether RtlZeroMemory zeroed a buffer correctly. It did not.

The failure needed both optimizations at once. The file system calls RtlZeroMemory, which is working through the buffer in floating point registers. An interrupt arrives mid-zeroing, and the driver’s interrupt path calls RtlCopyMemory, which uses the same floating point registers. The interrupt path never saved them, so when control returns to the interrupted zeroing, its register state has been silently clobbered and the buffer comes out wrong.

What it teaches

Two locally sound optimizations combined into a system-wide failure, and the person debugging it owned neither of them. OS development is full of cases like this, where modifying code in one place breaks code somewhere else entirely. A bug assigned to you is sometimes a bug in someone else’s code, and you still have to find it.