nib encoding: an oddity in the epilogue of the address field

[Bischoff]

Hello,

I stumbled today on something very weird.

"Beneath Apple DOS" says that the epilog of an address field is DE AA EB, and your code conforms to that :

• https://github.com/whscullin/apple2js/blob/master/js/formats/format_utils.ts#L164
• https://github.com/whscullin/apple2js/blob/master/js/formats/format_utils.ts#L262
• https://github.com/whscullin/apple2js/blob/master/js/formats/format_utils.ts#L389

But if you take an hexadecimal editor and analyze for example https://mirrors.apple2.org.za/ftp.apple.asimov.net/images/masters/blank.nib, you will see that there is DE AA directly followed by 5 bytes of gap FF FF FF FF FF. The EB is missing.

I traced what an Apple II+ ROM does when I save a file, and indeed it starts writing the gap 1 byte earlier.

I get an I/O error if I do a SAVE HELLO in an emulator, save the resulting bytes to a disk with the extra EB, and read the disk again. I don't get this I/O error if I remove the extra byte.

"Beneath Apple DOS" says about this byte "Oddly, the $EB is always written during initialization, but is never verified when an address field is read". I did not check what happens during an initialization. It could be that the $EB is overwritten at the first occasion, I do not know.

In any case, it seems to me that emulators work better if first epilogue is made only of 2 bytes, like what is found on all nib disks I checked so far.

In the hope it helps,

[whscullin]

This is very interesting. I feel like this may have come up when @iflan was writing tests for the various nibble routines. I will have to dig in some more.

[iflan]

Don't know how I missed this thread. :-)

@bischoff, I do remember reading that passage in chapter 3 of Beneath Apple DOS:

Lastly follows the epilogue, which contains the three bytes $DE, $AA, and $EB. Oddly, the $EB is always written during initialization but is never verified when an Address Field is read.

And I just checked Beneath Apple ProDOS, chapter 3, which says:

Lastly follows the epilogue, which contains the three bytes $DE, $AA, and $EB. The $EB is only partly written during initialization, and is therefore never verified when an Address Field is read.

In Understanding the Apple IIe, on p. 9-27, Slater writes:

An interesting point about DOS formats is the DE AA EB series that follows every address field and data field. Apple has always had trouble writing the EB. In RWTS 3.2, they cut off the EB at the end of the data field by neglecting to wait 32 cycles before switching READ/WRITE to READ after storing EB in the data register. They changed that in RWTS 3.3, so the EB is actually written at the end of the data field. However, RWTS 3.3 and the FILER formatting routine both cut off the EB at the end of the address field. Those cut-off EBs are not really written, and no attempt to verify their presence is made in RWTS or DIIDD processing.

I think this explains what's on blank.nib. Essentially, it's a blank, formatted disk that was written by RWTS 3.3. Any partial write of a byte would be discarded, leaving the five $FFs.

I'd very much like to see what happens with real floppies using both DOS 3.3 and ProDOS as well as factory-formatted floppies. I recently received an AppleSauce controller, so I should be able to see exactly what's happening. (This is going to take a few weeks, though, because I need to find a Mac to use the software.)

My proposal is this: If factory-formatted floppies contain the DE AA EB epilog for the address field, then I think it's reasonable to emit those bytes after the address field for .dsk files. Otherwise, we should do as @Bischoff says and omit the EB for the address field. If, eventually, we always use WOZ as the internal format, we can even emulate whatever portion of the EB actually makes it to the disk before it gets cut off.

[Bischoff]

Thanks guys for diving into this.

@iflan's proposal at emission time makes sense to me.

But I would also recomment to make the EB optional at reading time. If I am not wrong in interpreting the source code, it is currently expected when reading the disk.

[iflan]

@Bischoff, two of the three places that you identified above are only for "exploding" a .dsk sector into the .nib format that's used internally. The readSector method (disk2.ts, line 389) completely ignores the entire epilog for the address field—it doesn't even check the DE AA bytes. And, in any case, readSector is only used for exporting the disk state (DiskII.getBinary (used in the save dialog) and DiskII.rwts (unused)).

Therefore, I think that the emulator will be have correctly. If you think that's wrong, please let me know.

[Bischoff]

Therefore, I think that the emulator will behave correctly. If you think that's wrong, please let me know.

You probably know the source code better than I do. If it does not impact reading, then fine :smile_cat: , let's concentrate on the writing part.

[iflan]

To finish this off, I looked at a few professionally produced disks which I have imaged, including Ultima V and Chessmaster 2000, as well as personal disks that I'm pretty sure I formatted myself. All of the disks had the full DE AA EB epilog for both the address and data fields everywhere I looked. I did not do an exhaustive survey, but I tried to look at multiple sectors on several tracks for each disk.

Personally, I'm surprised that they all had the complete, intact epilog. But I guess that's good. :-)