Brother Tetris disk read issues when making interpreted image (brother120, brother240)

[barbeque]

We've talked about this briefly on Reddit private messaging, but to re-iterate:

Background

I'm trying to back up this single-sided floppy disk that came with my Brother "PowerNote" PN-4400 word processor, which I've just returned to the land of the living. Unfortunately, the floppy drive is not working right now (maybe broken belt?) so I haven't tested this disk to see if it works. I also have an unlabeled disk that probably has personally-identifying information on it, so I'm not comfortable dumping and sharing that one in public, we can work something out in private if it helps solve this mystery.

It's got a factory Brother label on it and contains the pack-in game Tetris as well as some other stuff (presumably spreadsheets, templates, etc.) I am fairly sure that this disk hasn't been formatted and rewritten on a word processor, so it should be as close to a "factory" alignment as possible.

My goal is to end up with a high-quality dump that can be uploaded to the Internet Archive and other sites for preservation, such that some weirdo in the future who finds a PN-4400 can make a Tetris disk using hardware like the Greaseweazle. I did something similar with the NEC mini5SX word processor's port of Tetris back in October, but that disk structure is very pedestrian.

For the Brother word processor, I did a flux dump (SuperCardPro/SCP format) with the Greaseweazle software originally, and ended up with something that looked like this:

The HxC disk image utility could not make heads or tails out of it regardless of which encoding mode I switched the disk track viewer into. Which isn't unusual, as Brother is not claimed to be supported by any of the HxC stuff. I can provide that image as well, but it's 40MB, so I decided it would be better to stick to the much smaller FluxEngine flux images as much as possible.

Dumping Process

After talking to you for a bit, I used the FluxEngine software's GUI front end to do a read in both brother120 and brother240 modes. I did a flux dump and what I'm going to call an "interpreted" dump.

There were a lot of red squares/bad sectors on the GUI while dumping these disks, and unfortunately I didn't think to capture a screenshot of what it looked like. The 240 view was "wavier" than the 120 view, and each had what I would consider to be a ridiculous and varying number of sectors, which makes me suspect some kind of alignment issue.

If the flux dump isn't enough to help, and if need be, I can re-run the process and take screenshots of the GUI view (which is very pretty - nice job.)

Interpreted Images

Looking at both of the interpreted dumps in a hex editor, I can clearly see some strings, so I think sectors have generally been dumped properly:

Both the brother120 and the brother240 interpreted disk images seem to be byte-identical for awhile. vbindiff notices an interesting divergence starting around offset $2400:

And maybe some kind of weak or poorly written bits around $3868:

A big chunk of text is missing from the brother120 image, so I am guessing that the correct format is 240K:

Unfortunately, both the brother120tool and brother240tool tools crash or bail out when trying to load their respective images in order to decode the directory structure:

% ./brother120tool ~/Documents/brother120.img 
Warning: file '1)' claims to be 82 sectors long but its chain is 1
Warning: file '@?X' claims to be 0 sectors long but its chain is 2
Warning: file '??? claims to be 224 sectors long but its chain is 1
zsh: segmentation fault  ./brother120tool ~/Documents/brother120.im

% ./brother240tool ~/Documents/brother240.img 
Error: Unknown image format.

I also read as an ibm720 disk but that produced a blank disk image.

Next Steps

I'll try to run a few back-to-back image runs with brother240, and see if the images are identical. If not, I'll clean the disk and drive and try again. It looks good through the shutter window, but "looks good" doesn't mean anything with magnetic flux.

Environment

• macOS 10.15.7
• FluxEngine GUI, built from source, commit d8528d889ad6c9520ef337dbc4e629248ff2eda3
• NEC FD1231H "IBM PC" high-density floppy drive
• Greaseweazle V4 hardware, works fine with the GW software

Attached images, both decoded (*.img) and flux (*.flux) in each of the two brother modes:

brother-images.zip

[davidgiven]

I have good news and bad news: the good news is that I've just fixed a bug which was causing all Brother decodes to be wrong. The bad news is that I discovered this while you were doing all that work... it's currently building in #630 waiting to be merged.

Using the fixed version, I can decode your brother240.flux fine:

$ ./fluxengine ls brother240 -f brother-images/brother240.flux 
 3.0: 239 ms in 60753 bytes
      no more data; giving up
  TETRIS.OP4       10545 S
  INTRO.SD2         8650 R
  ADDRESS.SD2       8481 R
  BUDGET.SD2        9683 R
  CHECKS.SD2        9732 R
  COMPOUND.SD2      8926 R
  EXPENSE.SD2       5240 R
  LOANS.SD2         5873 R
  SALES.SD2        12672 R
  SPLAN.SD2         8159 R
  SCHEDULE.DTB        78 A
  WP1AUSER.DCT      2182 HA
  WP2BABBR.PHR      1000 HA
(13 files, 91221 bytes)

You can try this yourself without waiting for the bugfix to roll out by adding --layout.layoutdata.physical.skew=1 to the command line. (What the problem is is that physical skew wasn't being handled correctly, resulting in all the sectors being in the wrong order.) I've hacked in a temporary fix and filed a bug to fix it properly.

[barbeque]

Brilliant, thank you for your super fast response!

I just pulled a fresh master (e3219087c98097e92aa3f4f18d99b513078c3100) and successfully extracted the TETRIS.OP4 file (I assume it's an executable) using the GUI in brother240 mode. I'll make sure to get this flux image up to the Internet Archive and move on to fixing the system.

Thanks again, always a pleasure.

[davidgiven]

What level of bad sectors did you get? The flux file you sent me is in pretty poor shape, and I think there's something wrong with the alignment. I see stuff like this in the log:

16.0: 239 ms in 60919 bytes
      no more data; giving up
      27 raw records, 13 raw sectors; 3.95us clock (253kHz)
      sectors: 13.0.0! 13.0.1! 13.0.2 13.0.4 13.0.5! 13.0.6 13.0.7 13.0.8! 13.0.9! 13.0.10! 13.0.11 14.0.0? 14.0.1? 14.0.2? 14.0.3? 14.0.4? 14.0.5? 14.0.6? 14.0.7? 14.0.8? 14.0.9? 14.0.10? 14.0.11?
      2816 bytes decoded

The ! means that it's seen a sector but it failed checksum test. The ? means that it was expecting to see a sector but didn't. It does seem to be expecting to see sectors on track 14, but has instead found sectors on track 13. This seems to vary across the disk, which is peculiar, but does suggest that the tracks on the disk aren't lining up with the head position.

There's a --drive.head_bias= which you can use to shift the position of track 0, and on brother240 disks it's normally 3, but I think your disk is happier with 2... but it's still not right. Your flux image seems to not have any retries in it either, which doesn't help. (Did you tell it not to retry? Hopefully that's not another bug.)

On an unrelated note... yes, the OP4 file is the executable; I haven't seen those before. It's a new Brother sub-architecture! Unfortunately the file is hopelessly corrupt. I actually have a CP/M port to a few Brother machines, but not this one. If you're looking for a project... http://cowlark.com/cpmish/arch/brotherop2/README.html

[barbeque]

Yes, things don't look good. I've tried cleaning the disk with 70% isopropyl and a foam swab, and the error is fairly inconsistent. Here are some abortive attempts at reading after successive cleanings of the disk and head:

I suspect the substrate is damaged, as I can mostly read out the other disk that came in the package except for the dirty part near the hub:

I do know someone who is talented at dumping disks, and I may send it to him (or you if you want!) but unfortunately I think this disk might just be damaged, which is a shame since I was hoping to preserve it. I used the default settings on the FluxEngine GUI regarding retries, but I will try the track shifting config in case it is some kind of timing misalignment, thank you.

edit: Now I’ve sat down and thought for a few seconds about what you were telling me, sorry about that. Timing misalignment feels more likely to me now that I’m looking closely at the picture. Comparing the two pictures, the third track of the Tetris disk seems to be “on the bubble” between a bad sector and the wrong sector (far right sector) and seems to be hallucinating an additional sector “too early” as well. Feels like it’s sort of reading half a sector and rounding up. Will try again with the head bias option next chance I get.

[davidgiven]

The extra sectors visible in the first picture indicate that it's seeing sectors that don't belong on that track. Once the image has been read any good sectors on the wrong track will actually be incorporated into the final image, but it shouldn't happen. You can, BTW, click on any track to get a visualisation of the raw data.

It feels very unlikely to me that there's anything wrong with the step between tracks, merely the position of track zero. So I don't know why some tracks are good and some are bad. Is there a possibility the disk is warped?

Brother FDDs, by the way, are capable of microstepping. The pinout is weird but one day I'd like to try and wire one up to a FE or GW. They're only single-sided but they sound like they'd be really useful for dealing with this kind of bad disk.

[barbeque]

Hi again David, some minor good news: the NEC PC floppy drive I had been using did a bunch of bad dumps on another dump project, then made a very bad burning-rubber smell which I think might be a bearing or o-ring on the spindle and seized up.

So I've grabbed another Mitsubishi drive out of my pile and decided to re-dump the Brother images with it. Still a lot of contamination on this disk, which I'll make another attempt to clean off (new swabs, new technique,) but early signs indicate that at least the "extra sectors" hallucination is cured.

The complaints on track 16 in particular are now gone, which is a promising sign. It now starts retrying and giving up around track 31.0.

[davidgiven]

Well... I've no idea why the NEC drive would have done that. But the new read does look better. There are still some extra sectors near the middle, however. Note that if you click on a track you can see all the details of what's actually going on down to the flux level, if you want to find out what those extra sector numbers are.