Formatting errors - BBC Micro 8271 controller

[kgl2001]

Sorry to be posting again, but it was brought to my attention that disk formatting is generating errors on the BBC 8271 based systems. This is not new to the latest 0.9.9a firmware. I've tested back as far as 0.9.6a and see similar errors. BBC 1770 based systems seems fine, though. Further details here:

http://stardot.org.uk/forums/viewtopic.php?f=16&t=13718&p=194925#p194925

[fordp2002]

I seem to remember there is a 4 sector interleave.

[keirf]

A visualisation of "blank_dsd - Corrupt.hfe" shows that FlashFloppy has taken a massive dump all over track 0 side 0. What a mess. I will have a closer look and also compare writes with the bitcell-offsets in the serial log.

[keirf]

@kgl2001 something is definitely wrong with the write alignments to HFE in this case. They're overlapping and just making a mess of track 0 side 0. I'm not sure why only that single track gets written to, as saving &4000 bytes should require writing to 7 track's worth of sectors. But perhaps things go off the rails and confuse DFS.

A test with a smaller write would be useful I think. Something even that modifies a single sector, such as OPT4,3, if that is enough to generate disk faults.

[PeterBBCUser]

Okay did a little test with a blank disk with the label "111111111111" I then did a *SAVE XXXXXXX 0 0 FFFFFFFF FFFFFFFF so a save of 0 bytes with load and execution address of FFFFFFFF I only did this once then when to look at the disk with disc doctor. Here is track 0 sector 0, it's what I would expect, it has written the file name in the correct place. Now track 0 Sector 1 has not been written to at all and that should have updated the write to 2 (5th byte) and put in the load and execution address for the file from 08 All the other sectors are readable on track 0 (contain ee) except sector 6 has an error and shows this Instead of ee. Hope this helps in trying to narrow down what is going wrong with BBC write. This was does with ver 0.9.11a and these track settings :- track-change = instant index-during-seek = no And this is the corrupt resulting file from the above test. Corrupt BBC test_ssd.zip Peter.

[keirf]

Thanks! I'll see what I can get from the HFE.

EDIT: It seems the write to sector 2 has clobbered sector 6 instead (they are adjacent, due to sector interleaving). Weird... Serial log could be helpful, though I don't know if you can readily set that up.

EDIT2: Please use v0.9.12a from now on, simply because it is latest released version as of today. So that is the best baseline for test.

[PeterBBCUser]

Sorry I cant give you a debug report I had ordered a RS232/USB adapter some time ago but it never arrived.

[kgl2001]

do you have a debug build of 0.9.12a that I can use for testing?

[keirf]

Here you go, 115200 baud debug build of 0.9.12a

ff_57_4.zip

[kgl2001]

Debug test 2 (write to disc):

• DNFS301 (DFS1.21) Installed on Beeb
• FlashFloppy updated to 0.9.12a (debug) firmware
• Flash Config in FlashFloppy wiped using two button reset procedure above
• FF.CFG file deleted from USB
• FlashFloppy & Beeb powered up.
• Created blank.dsd using DFSImager and converted to blank.hfe using HxCFloppyEmulator
• Write file to disc &10 bytes long - No disc error reported, but catalogue not correctly written

Attachments: blank_dsd - Clean.hfe : Clean blank copy of the blank_dsd.hfe file that is used for testing blank_dsd - Corrupt.hfe : Copy of the blank_dsd.hfe file after attempting to write &10 bytes write.txt : Copy of debug log during power up and write attempt

Test2.zip

Saving a file called '$.TEST' with a load address of &2000, and a file length of &10 bytes long:

Track 0, sector 0 bytes 8..15 (file name) seems to be written correctly:

Track 0, sector 1, bytes 8..15 (load address, execute address, file length & start sector) have not been written correctly:

I'll post up another test later this evening with a smaller file write that still gives a disc error. I think I was able to get disc errors if I tried to write a file bigger than about &400 bytes.

[keirf]

Thanks, the serial log shows DFS attempted to write to three sectors. That together with @PeterBBCUser's HFE image should be useful.

I am particularly worried because there's nothing all that FM/BBC specific in the HFE file. I wonder if I can repro this with a tailored write sequence on Amiga for example. That would make it much easier to fix!

[keirf]

I think in Test2 that DFS tried to write sector 0, 1 and 2. Sector 0: successful Sector 1: written too late, wiped out following sector IDAM/DAM (sector 6) Sector 2: written too late, wiped out sector+1's (sector 7) DAM and sector+2's (sector 3) IDAM/DAM.

First write to HFE bytes 167-232 = FM offset 668, length 4260 bitcells 2nd write to HFE bytes 3699-4764 = FM offset 14796, length 4260 bc 3rd write to HFE bytes 7231-8296 = FM offset 28924, length 4260 bc (All writes same length, each start is +14128 bitcells from the previous)

The correct FM offsets for these sector's data areas (DAM areas) are: Sector 0: 656 (off by +12 bits) Sector 1: 10512 (off by +4284 bits) Sector 2: 20368 (off by +8556 bits) (Error margin grows by +4272 bitcells per sector)

[keirf]

Ok, this is a bug in HFE writes since I implemented pipelined writes in v0.9.8a. I will sort out a fix for you to try.

[keirf]

Attached a fix for you to try. This is also a debug firmware, but no need to collect logs if it works!

ff_57_5.zip

EDIT: If it works, a further test is to try DFS 1.20 with the non-default track-change/index-during-seek options. It will probably fail with Fault 10 errors, and then we can categorically state that the correct setup for BBC & 8271 is to use DFS 1.21+ and out-of-the-box FlashFloppy, no weird options.

[PeterBBCUser]

writing a 0 byte file (*SAVE XXXXXXX 0 0) with this version (ff_57_5) on DFS 1.21 it wrote nothing to sectors 0 and 1 sector 3 has an error and I see data 00 00 06 01 sector 7&8 has an error and is over written from byte 4c with data 00 00 00 00 00 01 F6 62 62 62 62 62 62 62 62 B0 B0 B0 B0 B0 B0 B0 48 and the rest of the sector is filled with 00 all the other sectors on track 0 contain the normal E5 (except 0 & 1 of cause).

[keirf]

@PeterBBCUser thanks.

@kgl2001 a serial log would be interesting, no need to send HFE images.

I may need to construct a suitable test myself so I can repro.

[keirf]

Oh hang on, I can see the problem. I will work up another fix...

[PeterBBCUser]

okay still waiting for my USB to RS232 to arrive sigh Chinese been on holiday so it's still not shipped, so cant do a log as yet, but though I would test it and see if it was any better or worse.

[keirf]

Please try this new firmware:

ff_57_6.zip

[kgl2001]

That one seems to be working. I successfully saved a file &10 bytes long and read it back again. I'll try some further tests shortly...

Edit: Definitely moving forward, but I still see a problem. I can't save a file longer than approx &300 bytes. No error, it just seems to continually attempt to write. I'm still using DFS1.21 at this time.

So further testing. Writing &301 bytes fails, but &300 bytes works. The saved file starts at the beginning of track 0 sector 3, so it seems to fail when trying to write to track 0 sector 6?

[PeterBBCUser]

Yes I can write short files so definitely a step in the right direction, I can write lots of small files no problem if I try to save anything over &1000 bytes it gets stuck in some sort of write loop. i.e. if I try to *SAVE X 2000 3001 it gets stuck in a write loop (W flashing on the display in a pattern).

[keirf]

Serial log please, the written HFE might be useful too. No need for it to sit in the write loop for very long.

[PeterBBCUser]

if i use a openout and write one byte at a time it works , I think its when it goes over a track boundary with a save that it gets confused, hopefully kgl2001 can supply a serial log of whats going on.

[kgl2001]

Blank_300.hfe & write_300.txt : successful write of &300 bytes Blank_400.hfe & write_400.txt : failed write of &400 bytes

Test3.zip

I thought it was track boundary issue to begin with, but the &400 byte test I was doing is still all within track 0.

[PeterBBCUser]

Well I wrote a file (byte by byte) with openout till the disc was full and that was no problem, so it seems its larger chunks at once that causes a problem and have been able to write &1000 bytes to a file with *SAVE I can't quite work out what is going wrong but it can't be far off now.

[keirf]

Thanks I will look more closely tomorrow.

[keirf]

Seems that the &400-byte case writes the 4 file data sectors endlessly. It's hard to tell whether this is actually successful or not as it looks like you write a file of zeroes, and the empty sectors already contained zeroes? Perhaps you could write a pattern instead?

I will double-check the write offsets but I suspect they are fine and certainly the HFE images are no longer corrupted. One theory: lack of timely index pulse causes retry, endlessly? Weird that retry would be endless in this case yet Fault 10 could still be observed if seek times were slow (@PeterBBCUser).

If that's the case, need to make HFE writes faster, or support SSD/DSD direct ;)

EDIT: Checked the HFE files, they look fine, data updates are all in correct positions.

[kgl2001]

Yes. I can write a pattern of data to the disc instead of a bunch of zeros. I'll do that this evening when I get back from work.

[keirf]

Please use this firmware. Not expected to make much difference but it has various cleanups/fixes I've done today and it makes sense to be testing latest as I prepare to do another release with fixes so far. It's a debug build again.

ff_57_7.zip

EDIT: You may also want to break out of the write loop, perhaps power cycle, and see whether the file was in fact written ok despite the endless loop. That would be interesting. Another avenue of attack here would be to diff DFS 1.20 and 1.21, find which offsets differ, disassemble those regions, and see if we can work out where the retry loops are. They are likely much more optimal for read-after-seek (which affects real PC drives) than they are for writes and read-after-writes (which affect only FlashFloppy).

[kgl2001]

Here's another copy of the &400 byte write test. Using the latest debug build, and this time with a data pattern to make it easier to see what's being written. The start of every 256 byte block of memory has the text 'Sector x'. The write still hangs the computer, but I can now see that all 4 data sectors (3 thru 6) are being written to correctly.

Test 4.zip

[keirf]

Okay so if you load that HFE back into the BBC it will read that file correctly....

So the hang is presumably caused by drive-not-ready retries. The retry loop is long enough that you always end up with drive-not-ready status by the end of the loop (index pulses missed).

The easiest improvement is probably simply to support SSD/DSD directly (#16)

[kgl2001]

Correct. The file reads back correctly. Would it be worth trying in DFS 1.20 with the FF.CFG options set again for the BBC?

[keirf]

Since there is no seek or side change happening in your test, those options won't have any effect. I could give you a firmware which doesn't suppress index after write. It may yield sector-not-found errors instead but perhaps worth a try.

[keirf]

Here you go: ff_57_8.zip

[kgl2001]

That seems to work with DFS1.21 😃. I've been able to write &4000 (ie &10 x bigger) byte files without any problem! Just about to try a format now...

Edit: Still getting verify error with format, but definitely making progress!

[keirf]

Give it a kick around and with index-during-seek=yes too. I may hang it off the same option...

You might also want to try with that and track-change=realtime on DFS 1.20 too.

[PeterBBCUser]

Here is a image I formatted with the ff_57_8 version, whats is odd is it gives errors on almost all tracks while formatting but a verify afterwards read the tracks (till it hits where its gone complete wrong) without a problem, so seems a write/read together does not work as it should but a read alone does. I used disc doctor by the way to format this disc. The image shows the format verify output. Format_test_ssd.zip The image looks a lot healthier after a format than before, and on a formatted image I for some reason can save much bigger files. I also set these on : - track-change = realtime index-during-seek = yes Seemed to work better as well. O and writing a pattern, all my testing was done with ram filled with AA and if I did a dump the file was also filled with AA's

[kgl2001]

I'm using the same format utility that @PeterBBCUser is using, but I don't get the error on format that he gets. I do get the same error on verify. I've still to try it with the BBC options set and on DFS 1.20.

[PeterBBCUser]

WriteAA_test_ssd.zip Here is the same formatted image where i tried to save a file AA 1900 7C00 (&6300 bytes) but you will see where the AA stops and E5 starts is where it got stuck in a write/read loop (on ver ff_57_8).

[PeterBBCUser]

O am I am using disc doctor Ver 1.10 @kgl2001

[kgl2001]

@PeterBBCUser - Same here. Also using disc doctor v1.10. Perhaps a slow USB stick?

So, with ff_57_8 and the BBC specific config, I seem to be getting reliable reads and writes using DFS1.20 (&DFS0.98)! Still got an issue with the verify after format, but other than that, it appears to be working much better.

[PeterBBCUser]

I am using a "KINGSTON / Kingston DT Rubber 3.0 / 16Gb" with the following speeds : - Read speed: 189.26 MB/s Write speed: 33.79 MB/s (average) Which seem to also work better than the 1st one i was using which is :- Read speed: 19.74 MB/s Write speed: 6.17 MB/s

[keirf]

@kgl2001 so problem with verify after write, but the format actually went okay? (eg. all sectors present if you view the HFE image)

@PeterBBCUser your formatted HFE was missing a sector on track 35! I wonder what happened to it... All the others are there. Perhaps it was a write buffer overflow depending on USB stick speed. I'm afraid headline read/write speeds aren't much use, it's latency of small writes that really matters (512 bytes).

[kgl2001]

Format doesn't report any errors, but verify fails at track 6 (and other tracks, by the looks of it):

Test5.zip

Not sure what the blue line at the top signifies. I assume this is the index pulse? So occasionally losing a sector at the time an index pulse is generated?

Edit: Interesting. As @PeterBBCUser has highlighted below, I can then go and reformat the specific failed track, and that allows the verify to step onto the next error. By repeating this, I can then get a fully formatted and verified disc.

[PeterBBCUser]

The speeds fall right down with small files See test result here no idea what happened to track 35 seems to sometimes get its nickers in a knot. Just tried a test with ff_57_8 it formatted a disk showing no errors but then crashed on the verify on track 0 which i then reformatted the one track but then it kept coming up with a different track with problems, that is with track-change = realtime & index-during-seek = yes. and was also able to save a big file no problem.... now to try it with "index-during-seek = no"

[keirf]

@kgl2001 yes whenever you lose a sector it always seems to be the first on the track (the one at the index pulse). Maybe the write wraps round and takes out that sector's IDAM. I wonder why that would happen, if so...

EDIT: Could be the write continues to index pulse and that happens too late for some reason. I will have a think. Fair to say not many users are testing HFE writes, let alone HFE formats.

[keirf]

Do either of you have a logic analyser (eg. Saleae clone)? Just a thought, as it could be helpful to get a trace of (at least) the RDATA, WDATA, INDEX pins on the floppy interface.

I think I may need to sync INDEX pulses to the disk image better on writes (I do it on reads but INDEX is left "free running" off a timer on writes at the moment). I'll have to think how best to do that and how much work it is.

That doesn't really explain why format works for most tracks but not all, rather than consistently failing/succeeding. But at least it's a working theory.

[kgl2001]

Unfortunately I don't have a logic analyser (yet - I've just ordered one from China, though).

[PeterBBCUser]

Afraid I don't have a logic analyzer.

[drdpj]

I do, I'm just seeing if I can get it up and running. Had to build a new cable for extra probey options...

On 2 Mar 2018 07:28, "Peter" notifications@github.com wrote:

Afraid I don't have a logic analyzer.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/keirf/FlashFloppy/issues/57#issuecomment-369844170, or mute the thread https://github.com/notifications/unsubscribe-auth/ACdNPDzXn06IquiKXZxJkGKj1V4VQ9eoks5taPSMgaJpZM4SMvw1 .

[keirf]

Here is a firmware which should prevent a write from wrapping round and overwriting itself, by discarding the tail end of the write. This is a workaround for poor index timing, which is more effort to fix (will have to be done post v1.0). See how this one works for format.

ff_57_9.zip