Formatting errors - BBC Micro 8271 controller

[kgl2001]

Sorry to be posting again, but it was brought to my attention that disk formatting is generating errors on the BBC 8271 based systems. This is not new to the latest 0.9.9a firmware. I've tested back as far as 0.9.6a and see similar errors. BBC 1770 based systems seems fine, though. Further details here:

http://stardot.org.uk/forums/viewtopic.php?f=16&t=13718&p=194925#p194925

[keirf]

This is presumably with an HFE image, formatting single density (since it's an 8271)?

Can you attach a well-formatted HFE and one of these broken-formatted HFEs, along with any logs/screenshots of the format-verify errors?

[kgl2001]

Sorry for the delay in responding. Yes, I'm using .hfe images and formatting them single density. I create the .hfe image using HxC floppy emulator. During testing I've discovered a few other things:

1. I can't write to a newly converted .hfe disc on 0.9.9a at all. I get a disc error.
2. If I switch back to 0.9.6a I can write to the same disc, and then read back without any problem.
3. If I format a newly converted .hfe disc on 0.9.9a I get errors during format and a fail during verify.
4. If I format a newly converted .hfe disc on 0.9.6a I get less (sometimes no) errors during format and a good verify. Subsequent disc writes fail.

I've attached disc images for 2, 3 & 4 above. I don't have any logs and the screenshots don't really tell you anything more than I've written above. hfe disc images.zip

0.9.9a Format / Verify (item 3 above):

0.9.6a Format / Verify (item 4 above):

[keirf]

Thanks, perhaps you could try v0.9.7a too out of interest? No need to attach images etc for that one, just interesting to know how it compares goodness-wise with v0.9.6a.

[kgl2001]

0.9.7a also fails when I try to write to a newly converted .hfe image. I'm actually using the patched 0.9.7a file you provided to address the read issues on the BBC micro (ff_47_2.zip):

[keirf]

Ok thanks, I will have a poke at the HFEs and see if I can conclude anything.

[keirf]

Okay, first of all, try this firmware, which is a modified v0.9.11b. It will require the usual options: track-change = realtime and index-during-seek = yes.

I hope this works as well as v0.9.6a, and in that case we can dig into the format errors...

ff_57_1.zip

EDIT: Actually I think this one might solve all your problems. We'll see. :)

[kgl2001]

Thanks for looking into this for me. Unfortunately I'm still getting drive fault errors when trying to write. I'm also still getting the verify error after a format, but I don't seem to be getting the intermittent track ? errors during the actual format that I was seeing before.

[keirf]

On reflection I suspect the problem is still that the BBC is using index pulses for multiple purposes:

• To determine if drive is 'ready'
• To determine if a full revolution has been scanned
• To determine when to start track format

Since the firmware takes time to write back new data before it can serve up read data (unlike a real floppy drive) it disables index pulses during writeback phase. If it did not, host can time out on a following read when it sees index pulses while Gotek is not serving up read data. But this is a problem for BBC which constantly requires a supply of index pulses to avoid disk-not-ready error.

I don't know if this particular circle can be squared, I'm afraid.

Some thought:

1. This would be made worse by a USB stick with slow writes. What stick are you using, and have you tried more than one?
2. Is it possible to get serial logging by connecting USB-serial dongle while Gotek is installed? Also needs a debug build of the firmware (you can build, or I can supply). Don't worry too much if not, but this can be useful.
3. Ultimately the best solution might be to upgrade to DFS 1.21 which is also the standard solution for modern 3.5 drives. Are you able to try this? If you cannot make the ROMs yourself someone else on the BBC groups is bound to be able to, or possibly I have suitable EPROMs in stock and could sort it for you.

The benefit of (3) is that, if it fixes all the issues, we know for sure that all the firmware problems are to do with index-signal timing. And if it does not, we know there are further issues to address, possibly with FM data read/write (I think this is actually unlikely though).

Anyway, some food for thought there. This could be a can't-fix...

[fordp2002]

Is the processing power not there to emulate the drive at the same time as writing to flash or is it a lack of RAM?

[kgl2001]

With 0.9.6a, I do get some errors during format, (which don't show up during verify - so I think formats are actually working ok), but data writes are working fine, so hopefully it is fixable with the latest firmware!

1. I'm using a Sandisk Cruzer Fit 16 GB USB 2.0 drive. And yes, I have tried several others.
2. Yes I can do that, if you could supply the debug firmware, or give me some guidance on how to build myself.
3. I'm not familiar with DFS1.21. I assume it's a patched version. If I can find the ROM, I can install it on my system (either into flash or SWRAM) without any issue. I'll go searching...

Edit: Sorry, the above is not quite correct. I've flashed 0.96a, and I don't get reliable data writes, but I do get fairly consistent fails. On a blank disk (blank .dsd coverted to .hse) I can write a file upto &400 bytes long starting at t0/s2 and it works fine (90% of the time). If the file is bigger than &400 bytes it fails with a drive 10 error.

[keirf]

@fordp2002 Lack of RAM, need the whole track buffered to restart read while write-out is happening (as cannot read from USB stick while a write is in progress). Smarter use of available RAM (implement a proper buffer cache) can certainly help but I don't think can achieve perfection with 64kB RAM.

[keirf]

@kgl2001 So it is easy for you to try DFS 1.21? I strongly recommend if so, and try latest released firmware with following options settings:

track-change = instant
index-during-seek = no

[kgl2001]

Ok. Found a copy DNFS302, which includes DFS1.22. Installed 0.9.11a and applied the recommended options, but still get verify fail after doing a format. I assumed DFS1.22 inlcudes the necessary patches to work with 3.5" drives, but will now try DNFS301 which includes DFS1.21...

Edit: No difference with DFS1.21. As well as verify failing, I can't write to a .hfe file that's been converted from .dsd. I get a Disc fault 18 error.

I'm going to be away to watch Scotland beat England at the 6 Nations Rugby! so won't be able to do any more testing until I get back later tomorrow.

[keirf]

Okay, thanks for the testing so far! We'll try not to hurt you too much in the rugby ;)

I think we should stick with DFS1.21 and latest firmware, no extra FF.CFG options, and address the issues one by one.

I can send you a debug firmware if you can connect USB-serial dongle and collect logs from a remote terminal program? My usual baud rate is 3Mbaud but I can drop it to 115.2k for example if that works better for your remote host.

"Disc fault 18" is sector not found (http://chrisacorns.computinghistory.org.uk/docs/Acorn/AN/019.pdf). That's quite interesting... You definitely have 0.9.11a installed? That feels like the FDC timing out after seeing two index pulses after a write but no read data returned during that period. The modified firmware I posted here (0.9.11b) allows index pulses during writeback but that shouldn't happen in 0.9.11a.

[kgl2001]

I'm fairly certain it's the 0.9.11a firmware that's installed, but I won't be able to confirm until I get back. I definitely downloaded it, and stuck the .upd file on the USB stick.

I'm not sure if my serial dongle can work at 3Mb. I normally hook it up to my RPi at 115,200 and I know that definitely works. so probably best that I use debug firmware set at 15,200. I'll use PuTTy to gather the logs.

[keirf]

Okay no problem, I think if it is 0.9.11a and with the extra FF.CFG options disabled / returned to defaults, that we are returning to first principles on this bug. Let's see tomorrow.

[PeterBBCUser]

where can one find the DFS1.21 for BBC micro 8271?

O found it here http://mdfs.net/System/ROMs/Filing/FixDNFS/ I see there is also a version 1.22 :- DFS 1.22 patch for 3.5" drives and reduced workspace.

[PeterBBCUser]

This how a disc ends up looking after formatting with this basic program. Here is a copy of the formatted corrupt image. CorruptSSDDisc_ssd.zip This was with DFS 1.21 on a BBC with 8271 and track-change = realtime index-during-seek = yes remed out, does the same with them enabled.

[kgl2001]

I don't think REM out works. The parameters get read from USB and stored in NVRAM, so you need to force them back to default:

track-change = instant
index-during-seek = no

[keirf]

That's right, I was about to reply to say the same. It is a weird behaviour in some respects, perhaps seeing a FF.CFG should reset everything... anyway, at the moment it does not :)

Congratulations on the rugby :(

[kgl2001]

Thank you! We were long overdue a win! Good game - from a Scotland perspective 😃.

[PeterBBCUser]

Ahh okay was wondering about that.... anyway turning them off gives me the same result. did notice the format took a lot longer with paused in between with them off.

[keirf]

I think there is clearly some other issue than the index-timing issues addressed in #47. As a baseline can you confirm that DFS 1.21+ with latest firmware, no options, reads fine (ie. no regression on #47 using the appropriate DFS ROM and completely vanilla out-of-the-box FlashFloppy).

Following that, let's tackle writes first, then move on to format/verify. I need to know at the very least exactly what error message is produced by writes, whether it happens for only large writes, or all writes, or whatever. I can then also provide a debug firmware as discussed previously.

ie. we are doing a hard reset back to basics on this issue.

[kgl2001]

Out of interest, do you know if a flash update wipes he config settings or do they survive a flash? Perhaps it would be best to have FlashFloppy force the default setting or each power up, and then read any changes from the ff.cfg file?

[keirf]

A flash update (via USB stick) does not wipe the config. To wipe the config you need to press both buttons for a couple of seconds with no USB stick inserted (will see RST on 888 display, or more informative message on LCD/OLED).

Reason for flashing the config at all is to get certain options correct immediately on power-on, such as interface mode, font & other display settings, etc.

[PeterBBCUser]

with DFS 1.21and the following settings : - track-change = instant index-during-seek = no If it had to do a long seek to load the next bit i.e. I did a verify no problem but then tried to load a file which it had to then seek back to track zero I get "Drive fault 10 at 00/C1"

[keirf]

That does seem a bit surprising, Drive fault 10 means "drive not ready"[*]. And DFS 1.21 is supposed never to return that error but to retry until success or some other error[**].

Another page worth mentioning: http://www.sprow.co.uk/bbc/floppydrives.htm

It might be worth confirming on BBC groups that DFS 1.21 really does avoid all fault 10 errors.

[*] http://chrisacorns.computinghistory.org.uk/docs/Acorn/AN/019.pdf [**] http://mdfs.net/System/ROMs/Filing/Disk/Acorn/versions

[PeterBBCUser]

was reading that and then though hey you can change the seek times : - Step time: | Settle time: | Setting: 8271 | 4ms |16ms | SW3 closed SW4 closed (or *FX255,207,0 then press BREAK) with the faster seek times (you can hear it if you have the speaker connected to the gotek) it then does not come up with the drive error 10 problem. on my BBC I have the switches on the keyboard but they where open so have now set switch 3 and 4 on. I guess the Ver1.21 waits a bit longer but not long enough if the seek takes too long it still times out, though it does say it is suppose to wait till success or another error....odd.

[keirf]

That is mysterious. I suppose one thing to try would be to test DFS 1.20 with the faster seek times and see if that does still produce fault 10. That would at least confirm that 1.21 is doing something different than 1.20?

It's disappointing if 1.21 doesn't retry forever as the time taken to write data back to USB stick can vary and take a long time. That's a long time with no index pulses if DFS is (re)trying a read immediately after a write.

It might be worth contacting the (probable) author of the 1.21 patch? http://mdfs.net/User/JGH/

[PeterBBCUser]

1.20 does generate drive fault 10 even with the fast stepping. I am only trying to read as writing does not work and even with the fast stepping, which I did try.

[keirf]

Okay, so with 1.21 and the fast seek and doing simple writes (not formatting, yet), what errors do you see? Drive fault 10 I would expect?

This may be a case of needing to make writeback-to-USB go faster. First step might be to get a debug firmware and get serial logging as the writes happen, so that we get some timing info from FlashFloppy. And/or attach a logic analyser (eg. Saleae).

[PeterBBCUser]

I am not even trying to write any data to the drive, as you see here i do a CAT you see the contents, I try to do a OPT4,3 (change boot option) on 2nd DIR you see it has not even changed the boot option I try it a 2nd time and then get disc fault 18.

[kgl2001]

*OPT4,3 does actually write (or at least attempt to write) a small bit of data to the disc at track 0 sector 1. See 'Boot Option' in the following wiki:

http://beebwiki.mdfs.net/Acorn_DFS_disc_format#Disc_descriptor_fields

[PeterBBCUser]

Yes I know that but even a simple operation like that doers not work. it changes a bit on sector 2.

[kgl2001]

First test (read from disc):

• DNFS301 (DFS1.21) Installed on Beeb
• FlashFloppy updated to 0.9.11a firmware
• Flash Config in FlashFloppy wiped using two button reset procedure above
• FF.CFG file deleted from USB
• FlashFloppy & Beeb powered up.
• Select Time & Magik floppy (converted from .dsd to .hfe) - all files load without any errors. I used this disc as it uses most of the available space on the disc.

[kgl2001]

Second test (write to disc):

• DNFS301 (DFS1.21) Installed on Beeb
• FlashFloppy updated to 0.9.11a firmware
• Flash Config in FlashFloppy wiped using two button reset procedure above
• FF.CFG file deleted from USB
• FlashFloppy & Beeb powered up.
• Created blank .dsd using DFSImager and converted to .hfe using HxCFloppyEmulator
• Write small file to disc - no errors reported
• Try to read disc (*.) - Disk Fault 0E at 00/01

[kgl2001]

Third test (read from disc):

• Original DNFS (DFS1.20) Installed on Beeb
• FlashFloppy updated to 0.9.11a firmware
• Flash Config in FlashFloppy wiped using two button reset procedure above
• FF.CFG file deleted from USB
• FlashFloppy & Beeb powered up.
• Select Time & Magik floppy (converted from .dsd to .hfe)
• Try reading file !boot from disc - Drive Fault 10 at 01/C1 (consistently)

[kgl2001]

Forth test (read from disc):

• Original DNFS (DFS1.20) Installed on Beeb
• FlashFloppy updated to 0.9.11a firmware
• Flash Config in FlashFloppy wiped using two button reset procedure above
• FF.CFG added to USB with options track-change = realtime & index-during-seek = yes
• FlashFloppy & Beeb powered up.
• Select Time & Magik floppy (converted from .dsd to .hfe) - all files load without any errors. I used this disc as it uses most of the available space on the disc.

Let me know if you want any other tests done.

[keirf]

Thank you both for all these tests. Let's pursue the write errors with DFS 1.21 and FF v0.9.11a w/ no options. The error 0E (and quite possibly the error 18) is consistent with the write corrupting sectors on the HFE image.

Attached a debug version of v0.9.11a which logs to the serial line at 115200 baud. It would be helpful to run the Second test and collect logs, and/or send me before & after HFE images.

ff_57_2.zip

[kgl2001]

Can you clarify the wiring config for the serial interface. Do I use the same Tx / Rx / Gnd pins on the GoTek that I would use to flash the device? It's just that I can't get it to work using those pins. I've swapped Tx / Rx around, but no difference.

[keirf]

Yes that's right. Of course you do not need to supply Vcc if that is coming from the BBC.

[kgl2001]

No boot jumper required, I guess?

[keirf]

Correct. Boot jumper is to get you into programming mode.

[kgl2001]

Sorry. I'm struggling to get this working. I'm using the same FTDI adaptor that I used to program the unit in the first place, but not getting anything back to putty. I'll need to look into it further tomorrow. I'll verify the adaptor is working on my RPi first.

Edit: Serial interface is working fine with my RPi Zero @ 115200baud, so I'm not sure what's wrong with the GoTek. I assume I just need to do an update flash from USB on the GoTek? Or do I need to do a full reflash? Do I need to do anything extra to get debug working (eg any config options)?

[keirf]

Argh sorry that was not a debug build. This one is ('strings' on the .upd file and you will see the debug log messages embedded in it). An update from USB stick will suffice.

ff_57_3.zip

[kgl2001]

Ok. That's working now. Thanks. I really need to figure out how to build myself. I did see some guidance on the Wiki. I'll have a look at that later.

I don't have time to test right now. I'll do this when I get back from work this evening. One initial observation is that I get a readable data stream on initial boot of the drive, and when I swap discs, but nothing when I actually read the disc. Is that correct?

[keirf]

If you mean you don't get log messages synchronised with your actual read requests from the host system, that's normal. There is no explicit read request to a floppy drive, that is it's normal mode of operation (when not seeking or writing) and it is constantly streaming read data to the host.

[PeterBBCUser]

I am not 100% sure I am using the correct settings for the Acorn BBC micro, these are what I found hunting around on the Internet, could someone confirm that they are correct for BBC .ssd disk image. Thanks Peter.

[kgl2001]

Debug test (write to disc):

• DNFS301 (DFS1.21) Installed on Beeb
• FlashFloppy updated to 0.9.11a (debug) firmware
• Flash Config in FlashFloppy wiped using two button reset procedure above
• FF.CFG file deleted from USB
• FlashFloppy & Beeb powered up.
• Created blank.dsd using DFSImager and converted to blank.hfe using HxCFloppyEmulator
• Write file to disc &4000 bytes long - Disk Fault 18 at 00/02

Attachments: blank_dsd - Clean.hfe : Clean blank copy of the blank_dsd.hfe file that is used for testing blank_dsd - Corrupt.hfe : Copy of the blank_dsd.hfe file after attempting to write &4000 bytes write.txt : Copy of debug log during power up and write attempt

Test.zip

[keirf]

That's a great test thank you. I will investigate tomorrow.