Differences detected in ADF files

[ourIThome]

So … I’ve been using “Photon Paint” as my test disk.

Photon Paint v1 – This was using the older build which produced 16 bytes too many at the end. The disk imaged perfectly, no need for fixing. Photon Paint v2 – using 385 build … I had some issues reading the same disk, tried some –retries / --revolutions to fix, and ended up adding some isopropyl to fix. Photon Paint v3 – using 385 build … same as before, some errors reading the disk again, --retries / --revolutions to fix, and again ended up using some isopropyl to fix.

So, v1 – this matched a previous adf capture using kryoflux – bar the 16 bytes at the end. I did a flexhex file compare of v1 and v2 … and I can see some differences. I did a flexhex file compare of v2 and v3 and again, more differences.

I use another tool called ADF Workshop – very handy when dealing with Amiga DOS files, to find any bad files / CRC problems.

V1 – I’m not bothered about the first file – I had the same when I did it in kryoflux. V2 – now PhotoPaint file has corruption / sector 566 V3 – Photopaint corrupt – 00 filled, sector 887 reported problems

I am guessing there is something I’m doing, that is detecting the flux as being good … could the isopropyl be skewing the results / detecting as no bytes?

I have shared Screenshots / 3 files (flux / adf / csv) directly with author.

If I discover any additional tests / ill add them here.

[ourIThome]

Not sure whats going on ... tried to re-re-read this disk, had to revery to isopropyl as the disk was getting noisy ... and now this adf matches that of the first one (without the additional 16 bytes). Im hoping the 3 flux files might shed some light on the differences?

btw, forgot to thank you for the fix for the 16 byte problem.

[ourIThome]

I did another 6 or so backups of the same disk / isopropyl -- all of them are now matching the same file CRC hash... I cant replicate my issue. However, if the Flux files shed any ideas ... they are there for you to take a look. If you dont see anything obvious ... feel free to close. Dont spend too much time on this, unless you discover a gremlin.

[davidgiven]

Wow, thanks! That's a really serious bug which I've noticed when writing disks, and I thought this was to do with the writer code... and it's not. The flux files point straight at the problem. The good news is it's a decoder problem rather than a sampling problem, so any flux files you have will be correct (and just need re-decoding).

[davidgiven]

Actually, I'm not sure that is the problem I'm seeing, but it is indeed a problem. What's happening is that sector 13.00.06 is bad. The sampling just happens to cut off in the middle of the sector header, and purely by chance the sector header checksum is valid. What FluxEngine typically does here is to pad the raw sector data with zeroes until it's the right size, as it makes the logic easier --- but there's a flaw in the Amiga checksum routine where a zero byte in the data doesn't cause the checksum to change. So, the padded sector passed the checksum and is considered valid data.

I don't think this is the same bug I'm seeing elsewhere, but only the Amiga with its very poor checksum algorithm is susceptible --- on all other platforms the sector will fail the checksum and be considered bad. But it still needs fixing.

[davidgiven]

Unfortunately this doesn't solve the different between images 1 and 2. There are two bad bits, but unfortunately they're both in the same sector and cause a 0x15 to be read as a 0x55 (or maybe the other way round). Again due to the Amiga weak checksum, these two errors cancel out and the checksum passes causing the sector to be considered good. A real Amiga would suffer from this too.

I may be able to do something about this; let me think about it.

[davidgiven]

I've merged the simple fix --- try release 389. This will fix image 3 but not the other two. As a workaround, crank up --revolutions to 2 or 3 and undetected bit errors should be reported as conflicts (C), assuming they're spurious and not actually on the disk.

[ourIThome]

Cool, Ill take a look this afternoon ... but I suspect it was a lucky catch that I managed to image a few flux images with differnt ADF results. But ill try my best and report any findings. Not sure if there is something more you need to do or planning on this issue ... but thanks for taking a deep look.

I know there was another ADF reader : http://amiga.robsmithdev.co.uk/history

I think he talks about something that you cant have two 11 bits in MFM encoding together... could something similar be done for flux, so it assumes there are multiple 1 bits next to each other? Or is this now fixed long term.

Anyway, makes for an interesting read.

[ourIThome]

Hi,

Shared some flux files privately to you – no rush

My initially testing with Photopaint, looks to be good – although its either working, or I am not able to replicate the problems I had yesterday.

I moved my testing to another disk : Nebulus

In the folder there is :

1) Nebulus.flux / adf / csv 2) Nebulus - 389 - Copy 1.flux / adf / csv 3) Cylinder 1.1 -- eject during reading/Nebulus - 389 - Copy 1.flux / adf / csv

In 1) above, this is a previous capture – which contains the extra 16 bytes at the end – seems to be good to read in ADF workshop – all files match CRC checks In 2) Build 389 -- this took a lot of reading over and over with lots of isopropyl, and kept getting the same results :

H.SS Tracks --->

0. 0 ................................................................................
1. 1 ................................................................................
2. 2 ................................................................................
3. 3 ................................................................................
4. 4 ................................................................................
5. 5 ................................................................................
6. 6 ................................................................................
7. 7 ................................................................................
8. 8 ................................................................................
9. 9 ................................................................................ 0.10 ................................................................................
10. 0 ................................................................................
11. 1 .B..............................................................................
12. 2 .X..............................................................................
13. 3 ................................................................................
14. 4 ................................................................................
15. 5 ................................................................................
16. 6 ................................................................................
17. 7 ................................................................................
18. 8 ................................................................................
19. 9 ................................................................................ 1.10 ................................................................................

I got bored, so a few hours later … tried a read, and then added some isopropyl and eventually I got a “good” read. However, if I load this in ADF Workshop – I can see the file “2” and “nebulus.code” are conflicting CRC – these files in 1) above are fine.

Another test I did, was whilst reading the disk, sometimes lifting the disk in the drive (assume to get closer to the read heads) has resulted in kryoflux getting good reads. Unfortunately, I am using the Panasonic drive which isn’t good for this method, and when I tried it … it showed good read. I don’t know if you can detect for this one … but thought id add it to the list of read problems. Not sure you can do anything – but shared with you.

[davidgiven]

Whoops, completely forgot about this.

I had a look at the two flux files, using the current head of git --- they both decoded fine with ./fluxengine read amiga. I can see that the images are different, but nothing looks obviously corrupt. I found an Amiga emulator and Nebulus ran fine from both disks.

I suspect that since you read these, I managed to bugfix the client and it's now producing better decodes. Could you try again with the latest release?

[ourIThome]

No worries, I know you have a million and one things to manage ... so happy that you can take a look when you get some free time.

I've just tried with build 434, I get the same results with the decoding ... all looks good as before. If I try to load nebulus (F1 Key) I get a read/write error that I can cancel and the game continues to work. If I press F2 -- Virus (game) -- I get a read/write error ... If I open up the Nebulus.adf -- which was previously ripped with kryoflux, these problems do not exist.

If the disk is a DOS Amiga ... I tend to use "ADF Workshop" -- within the sector data, there is a CRC32 for each file ... which checks for good / bad files. Sometimes bad files come from historic issues with the disks ... but as I know I have 1 good adf rip from kryoflux, I am pretty sure something is a miss with the flux files.

I have loaded the two flux captures into ADF workshop and I see :

The flux in the "eject folder"

---

Content list	Size	CRC32	BA	HR	SQ	Date	Information
2	102.588	00000000	OK	OK	KO	1987.09.12 10:09	Sect 836 seq 36 => Data block corrupted
nebulus.code	297.374	72614411	??	KO	OK	1987.09.12 10:17	Sect 175 => File Extension damaged

The flux in the "nebulus folder"

---

Content list	Size	CRC32	BA	HR	SQ	Date	Information
2	102.588	00000000	OK	OK	KO	1987.09.12 10:09	Sect 836 seq 36 => Data block corrupted
nebulus.code	297.374	72614411	??	KO	OK	1987.09.12 10:17	Sect 175 => File Extension damaged

I've trimmed out all the files reporting "ok".

The CRC files above report as "00000000" possibly due to corruption in the sector data / not being able to retrieve the expected CRC value?

Let me know if there is anything more I can do to help.