Reconstructing a lost NES game from 30-year-old source code disks

by Rich Whitehouse June 1, 2020
Video game programmer Chris Oberth (1953-2012)

Hey all, VGHF founder Frank Cifaldi here. Today I’m tag-teaming with Rich Whitehouse to bring you the story of how we recovered and re-assembled Days of Thunder, an unreleased, never-before-seen title co-authored by Chris Oberth at Mindscape.

Programmer and designer Chris Oberth had a long, diverse career in the video game industry before he passed away in 2012. I think for most of us, the definitive Chris Oberth game was Anteater, the arcade title he designed and engineered during his time at Stern (or perhaps you know the clone that Chris himself wrote for home computers – Ardy the Aardvark – or the one he didn’t – Oil’s Well). For others, it might be Winter Games for the Commodore 64, or perhaps his later arcade work at Incredible Technologies like Time Killers or World Class Bowling, or the handful of titles he developed on his Apple II in the late 70s and early 80s (on an Apple II with serial number 201, as he liked to brag).

In early 2020, The Video Game History Foundation was approached by a friend of Oberth’s surviving family to help them make sense of the materials he had left behind. Untouched for years in the home basement he often worked from were piles of old computers, CD-R backups, floppy disks, notes, cassettes, EPROMs, and data tape going back to his earliest Apple II work in the late 70s, which his family agreed to loan to us for evaluation.

As is often the case when looking at archives like this, our first priority was trying to recover any works of Chris’ that were lost – games he developed that, for one reason or another, were never available commercially. Unfortunately we found almost no remains of his time at Stern, which meant that his unreleased Hunt the Wumpus-inspired arcade game Crypt is likely gone forever. But the rest of the collection gave us high hopes for his later home computer and console work.

The mysterious “Hot Rod Taxi” was our first indication that Oberth’s code archive might contain unpublished work.

One disk in particular jumped out at us – a 5.25” floppy with a hand-written label that said “NINTENDO HOT ROD TAXI FINAL.” This is not a known title for a game, but we knew that Chris had developed at least one game for the Nintendo Entertainment System – American Gladiators. We didn’t know much else about his career around this time, but fortunately Oberth conducted at least two interviews in his later years with early retro game enthusiast websites – one with Derek’s Basement Arcade and a three-parter in 2006 for Retrogaming Times (specifically in issues 21, 24, and 27 of “Retrogaming Times Monthly”). It was in the latter that Chris briefly mentioned an unreleased NES game – an unpublished version of Days of Thunder done for Mindscape (a separate attempt done by Beam Software was ultimately published in its place). Was “Hot Rod Taxi” our lost Days of Thunder? Perhaps stripped of its movie license title?

Hot Rod Taxi for the NES, as compiled from Oberth’s source code. This isn’t a game, so much as an early tech demo. Days of Thunder still eluded us.

The answer turned out to be…sort of. Maybe. After making a clean flux rip of the disk and assembling a binary from its source code and graphics data, we were left with a playable NES game that, while historically interesting, was not much of a game yet. If anything, it appeared to be early proof-of-concept work, perhaps Oberth getting his feet wet for the first time on a platform that was brand new to him.

We had one other place to look: a giant pile of mostly-labeled hard drive backups, spanning several years, across nearly 40 floppy disks. These weren’t merely disks on files, they were done with (multiple) hard drive backup tools, meaning files were split and encrypted. The only way we were going to recover anything on these was to digitize the entire stack and then re-assemble them afterward.

From here I’ll kick it over to Rich.

Just a sampling of the hundreds of disks left behind by Chris Oberth. Is Days of Thunder somewhere in this pile?

Storage Recovery

Our first task in finding what remained of Days of Thunder was to recover as much data as possible from the available storage media, which consisted primarily of old 5.25” disks. Some of those disks had unreadable sectors, but many still were fully-intact. Some of the disks were also immediately readable, with FAT12 file systems created under MS-DOS. The most promising set of the bunch, however, was a collection of 21 disks containing backups of 4 separate hard drive partitions, in a proprietary format generated by PC Tools 5.1. We had some help figuring that part out, “PCTools 5.10. Backup” was written on the labels.

Our first attempt to reconstruct Oberth’s 1989 hard drive using his original backup tool proved unsuccessful, as emulation through DOSBox was not up to the task.

Initially, I tried running the PC Tools backup program via DOSBox in hopes that I might be able to restore the backups through emulation. I didn’t have any luck getting the program to recognize the disk images under DOSBox, and figured I might have to make a few changes to the emulator itself in order to go that route. At that point, I decided to take a detour toward understanding the data directly, and set about digging through the raw sector data in those backup disk images.

The structure of the data wasn’t too complicated. Every directory and file record was stored in a packed stream, where command bytes were used to indicate 1 of 3 things. 0xDA means “directory record”, to be appended to the current directory stack. 0xDB means “file record”, immediately followed by the actual file data. 0xDC means “pop directory”, which removes the most recent directory from the directory stack. Within the records, I was also able to identify MS-DOS style date/time words, which gave us the ability to put precise dates on each of the files in the backup.

This was a promising start, and allowed us to generate a list of files contained in the backups. We quickly spotted a lot of source code and data that seemed to almost certainly represent the big prize: Days of Thunder. Unfortunately, there was still a problem. Even though I was able to generate a full path list from the backups, most of the data we were after was compressed using a proprietary algorithm. It may have been easy enough to eyeball the compression used here, but encryption was also applied on top of the already-compressed data. Fortunately, the encryption applied to the data was generic and not based on a user-specified key, or I’d likely still be chipping away at recovering the data instead of typing this!

So from here, I considered a few approaches. The first approach was to create a custom build of DosBox in order to massage the PC Tools backup program into functioning correctly under emulation with the original backup images. The second approach was to reverse engineer the encryption and compression used by PC Tools, which would mean taking a good bit of time to disassemble (and, ideally, debug) the original program. I’m no stranger to disassembling systems more vast and complicated than this, but I figured a custom build of DosBox would be a useful tool in this approach either way, so I would only need to go this far in the event that the backup data was corrupt in some way. In that case, I’d have to understand every bit of it in order to salvage everything possible from the wreckage.

VGHF volunteer Foone came to the rescue, reconstructing an era-appropriate hardware environment to properly recover the backup of Oberth’s hard drive.

The third approach was a lot easier than either of the above: Use real hardware! Unfortunately, I didn’t have the necessary hardware on hand, but we were able to immediately enlist the help of someone who did. Foone was able to get a hardware-based solution up and running, and although we did have to re-rip one of the backup disk sets to get PC Tools to swallow it, we were ultimately able to restore every bit of data on each of those 4 hard drive partitions.

Because the third approach worked, I never did get down in the trenches of reverse engineering the compression algorithm used by PC Tools. Anecdotally, however, it seemed likely to be something LZSS-like. I did some simple tests using pcsecure.exe from the PC Tools collection, which appeared to use a compression very similar (although not identical, based on results achieved by compressing from matched data) to the compression used by the backup program. I also noted that the compression fell over and produced larger end results for some of the cases that you’d normally expect to be bad for LZSS, such as a series of completely unique bytes, producing the kind of overhead that I’d expect from peppering the data with control bytes.

Building the Source

Upon first glance, we didn’t see anything that stood out as being a pre-built ROM image. Of course, that would’ve been too easy. So with 4 hard drive partitions worth of data to sift through, I started in a directory named C:\ROMX\BTR, which is the directory we were most excited to see back when all we had to look at was a list of filenames. This directory seemed to contain all of the source code, and some data, for Days of Thunder. Many of the tools originally used to develop the game, including the assembler/linker, were also backed up in C:\UTIL. The original assembler was X6502, a 6502 cross assembler by 2500 A.D. Software, Inc. Both the assembler and linker here were marked as Version 4.03b.

I built the source and linked all of the objects together, which generated 6 separate binaries. This brings us to the next part of the process, a program called ROMX. This program is part of a ROMulator package by GTEK, Inc. The company’s website, really quite remarkably, is still up as of this writing. In our case, the ROMX program was used to communicate over the PC’s RS232 port, placing data directly in a NES development cartridge’s PRG/CHR memory. This allowed developers to quickly update and test their code and/or graphic changes on real hardware. Even more conveniently, they were able to selectively update only a particular set of banks in order to reduce iteration times when updating a particular piece of code or set of tiles. There are quite a few batch files in the source which run ROMX commands to upload specific files to specific places in PRG/CHR. Here’s one which tells us exactly how the entirety of PRG ROM is laid out:

ROMX PB0.TSK[TS,M\,%%0000,@0000-ffff]
ROMX PB1.TSK[%%0000,@4000-7fff]
ROMX PB2.TSK[%%0000,@8000-bfff]
ROMX PB3.TSK[%%0000,@C000-ffff]
ROMX PB4.TSK[%%0000,@10000-13fff]
ROMX PB7.tsk[%%0000,@1C000-1ffff]
ROMX CRASHL2.SND[%%0000,@1F000-1ffff]
ROMX SKIDF.SND[%%0030,@1FC80-1fff0,TE]

The above tells us where to place the 6 code binaries (the ones I just talked about linking together earlier), along with 2 pre-generated sound binaries. When one of the developers wanted to produce a flat binary from PRG or CHR memory, instead of replicating all of this information in another PC-side process, they would just run ROMX with parameters to dump all of the data right from the hardware and store it to a file.

At this point in building the game, it was clear that we’d need some kind of mapper based on the number of PRG banks alone, so I went looking through the source and saw some code which was explicitly working with MMC1 registers. I constructed an iNES header with my 128KB PRG ROM, set the mapper bits accordingly, and just threw some random garbage into CHR ROM for testing. The game successfully booted up in an emulator, and started playing music over a big mess of random tile data which I figured was probably supposed to be a title screen.

Missing Characters

From running the game with a bunch of random data in CHR ROM, I could see that we were looking at quite a bit of unique tile data. Some traces of this data remained in and around the source. In particular, there was a custom tool designed to parse tile blocks out of Pictor CLP files, write them to a temporary file, and upload them to explicit CHR addresses via ROMX. This process was being invoked for some small car and wrench tile sets. For the most part, though, a whole lot of CHR data was unaccounted for, and there was no sign of how that CHR data was normally uploaded via ROMX.

Up in another directory, C:\ROMX\CHEDIT, I found a custom NES tile editor with remnants of data which appeared to belong to Days of Thunder. I was able to match some of the .map and .pal data which had been exported here to data which was being assembled into PRG, and because that matched, I figured the .chr files in here were also going to be a match to whatever belonged in CHR. I started with a file named CAR.CHR, and set about figuring out exactly where it belonged in CHR memory. The answer to that was 0x3000, which I managed to verify by looking at the title screen even with the rest of CHR filled with random data:

We located and compiled the program data, but most of the art was missing! To have come this close and not quite crossed the finish line was very frustrating.

It was good to verify that this particular data belonged in CHR ROM, but I was still missing lots of data, and I’d discovered some references in scripts for an animation utility which pointed to track and tile images on drive paths which didn’t actually exist in the backup we’d restored. I feared the worst, which was that we were missing a whole bunch of CHR data along with the source data to reconstruct it.

Knowing that I already had a likely candidate for a nice big chunk of CHR ROM, I took a unique string of bytes from CAR.CHR and binary searched for it in every file in the backup, hoping I’d get lucky and hit a pre-built chunk of CHR ROM. No luck on the first attempt. At this point, I went for the nuclear option. I extracted every archive in every known format in every one of the drive backups, and did the same for every other disk image that we’d managed to recover. I ran another search over the whole thing, and there was a single hit on a 128KB binary file. Oh yes.

Putting It All Together

I was cautious, not knowing if that conveniently-sized 128KB binary file would actually match the source and tile maps I was working with, particularly because the hit came from a separate set of disks. Seeing no other option at this point, though, I took the file and used it to occupy the whole of CHR ROM in my constructed NES ROM, and there it was. Everything seemed to be exactly where it should be!

At last, the long-lost Days of Thunder, seen here for the first time in over thirty years!

With that, we’d succeeded in our mission. It does seem that we’re missing a lot of source data, unfortunately, and I still suspect that said data lived somewhere on those other hard drive partitions which weren’t included in the main backup we restored. If not for the fact that we managed to get a hit on the CHR binary from a seemingly unrelated set of data, we may never have been able to restore this game in its entirety, or we might’ve ended up authoring our own tiles just to see some semblance of the intended playable experience. It just goes to show, every floppy matters!

Build And Play It Yourself!

With permission from Chris Oberth’s family, we’ve uploaded the source code for Days of Thunder – along with a custom build tool, and pre-built binaries – on GitHub.

You Made This Possible!

We were able to spend weeks focusing on Oberth’s archives thanks to your generous charitable donations! This level of work really wouldn’t have been possible without people just like you. If you’re new to the VGHF, or if you just appreciate the work we’re doing, consider giving if you can. Every dollar helps.

We’re preserving the history of video games, one byte at a time.Donate Now