ibm.py read address mark failure needs accurate diagnostic data

[crestr1]

Have an assortment of disks that have different side 1 sector id's the IBM std ones are from 27. I have non std disks would like to know what is expected from the printout which may be expecting interleaved numbers just looking at the data printout. It reads standard ibm ones without error side 1 sectors are 27 up but this failing disk reads perfectly in ibm.scan mode

https://github.com/keirf/greaseweazle/blob/b0af2e4a5854ca13db7d6d23e84d77ad7b0a07c1/src/greaseweazle/codec/ibm/ibm.py#L658 line 658 to print statement Format dec.rx01 T0.0: IBM FM (26/26 sectors) from Raw Flux (222540 flux in 674.79ms) : Is ok side 0 T0.1: Ignoring unexpected sector C:0 H:0 R:22 N:0: now errors from side 1 follow T0.1: Ignoring unexpected sector C:0 H:0 R:13 N:0 T0.1: Ignoring unexpected sector C:0 H:0 R:24 N:0 T0.1: Ignoring unexpected sector C:0 H:0 R:4 N:0 T0.1: Ignoring unexpected sector C:0 H:0 R:26 N:0 T0.1: Ignoring unexpected sector C:0 H:0 R:15 N:0 T0.1: Ignoring unexpected sector C:0 H:0 R:6 N:0 T0.1: Ignoring unexpected sector C:0 H:0 R:17 N:0

my diskdef entry disk dec.rx01 cyls = 77 heads = 2 tracks 0-76.0 ibm.fm secs = 26 bps = 128 gap3 = 26 rate = 250 rpm = 360 end tracks 0-76.1 ibm.fm secs = 26 bps = 128 gap3 = 26 rate = 250 rpm = 360 id = 27 # normal for ibm side 1 sector nos? h = 0 end end

[crestr1]

dump of end of failing read.txt

gw read myimage.img --diskdefs /home/ubman/Greaseweazel/greaseweazle-1.19/src/greaseweazle/data/diskdefs.cfg --format dec.rx01 --revs 4

[crestr1]

More data from my tests - simplified my layout - data attached in zip cmd: "gw read motst.img --diskdefs /home/ubman/Greaseweazel/diskdefs.cfg --format mot.mdos3 --revs 4 --adjust-speed 360 --retries 0" Recovery screen.zip It is going after more sectors than it should: see attached. it has totally lost the plot for specs of side 1: should be 26 128 byte sectors/ track, 77 tracks.It seems to have got into some interleaved mode for 52 sectors/tk, none of which it can read. The good thing it shows the error marking on dud sectors works nicely in the .img file and it is bad-marking the correct no of bytes (0xD00) for the track it is not able to read. Doing all this under Mint Linux now.

[crestr1]

While the ibm.scan mode works and makes usable .img files with perfect disks. I has no way of handling defective sectors, it merely omits their length from the final .img. The important thing to me is needing to make complete intact .img files that just have a dummy value placed in bad sectors so the images are full sized and usable under an operating system's tool chains, the potential offsets to data pointed to by directory entries needs to be always preserved in the .img size structure.

[keirf]

Yes you will need to create a diskdef format for each disk format in your collection so that your reads can be checked against the format. In the original post here looks like perhaps your dec.rx01 format is not being picked up. Did you specify diskdefs=path/to/your/diskdef/file?

[crestr1]

Thanks Kier, The attached recovery_screen .zip contains the latest results cmd file and diskdefs that all play together. I have narrowed my collection down to two disk types the ibm ones that recover perfectly if ok (S1 starts at sector 27) and the others that are all the same but i have no way of describing. Is there a way I can set things to ignore the sector numbers and just recover the side 1 data sectors sequentially. this is I assume is what the ibm.scan mode does. but it still needs to know the size of the image it needs to create or it will screw up on bad sectors. Can you suggest where i can put print statements in the code that actually executes. the messages "T0.1: Ignoring unexpected sector C:0 H:1 R:10 N:0" is totally uninformative, what is R (expected or read from disk) and what is N which is always 0 ?

[keirf]

ibm.scan scrapes all the valid sectors it can find and writes them to an IMG file in ascending order of sector id.

The message comes from https://github.com/keirf/greaseweazle/blob/master/src/greaseweazle/codec/ibm/ibm.py#L681

You can actually run Greaseweazle out of the source directory by installing it with the -e option, like this, from the Greaseweazle source repository (eg. gw clone https://github.com/keirf/greaseweazle && cd greaseweazle):

python -m pip install -e .

With the above, you can run gw as normal but Python will be executed out of the source tree.

In this message, R is the sector id, N is the sector size (size is 128<<N, so N=0 means size 128 bytes).

[crestr1]

Hi Kier, Is there a way in the diskdefs I can set things to ignore the sector numbers and just recover the side 1 data sectors sequentially. Could you answer this please I'm having trouble following the manifestation of the variables: the messages "T0.1: Ignoring unexpected sector C:0 H:1 R:10 N:0" : what is R (expected or read from disk, you don't say) and what is N's expected derived from which is always 0 should this be so ? I sent in the zip the results I've gotten that suggest something goes wrong when one tacks the side 1 control on the end of the diskdef entry. I put 27 in id and it only works on IBM disks but I cannot find a number that works on at least one s1 sector of my other disks. This is strange. every sector number is ignored :

T0.0: IBM FM (26/26 sectors) from Raw Flux (216219 flux in 667.16ms) T0.1: Ignoring unexpected sector C:0 H:1 R:10 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:19 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:1 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:12 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:23 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:14 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:25 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:16 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:5 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:3 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:7 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:18 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:9 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:20 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:11 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:22 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:2 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:13 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:24 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:4 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:15 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:26 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:6 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:17 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:8 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:21 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:10 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:19 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:1 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:12 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:23 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:14 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:25 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:16 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:5 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:3 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:7 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:18 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:9 N:0 T0.1: Ignoring unexpected sector C:0 H:1 R:20 N:0 etc it's all in the zip i sent and its repeated twice even though the retry is off in the CMD could be that the derivation of what is expected gets screwed up by what is on the disk which almost seems to be interleaved by 11 or in some strange way if it is its sector numbers being shown. This then said, why does the ibm.scan recover the data perfectly to an .img

[keirf]

CHRN are the sector header bytes. C = Cylinder, H = Head, R = Sector Number, N = Sector Size (128<<N).

In this case the correct diskdef would be the standard single-sided dec.rx01 as shipped with Greaseweazle, except with heads = 2.

[keirf]

If you just want to recover any sectors found, use format ibm.scan.

[keirf]

Apologies. Please share a raw dump of an offending disk (eg. some variant of gw read name.scp) and your attempted diskdef configuration, and I can advise how to fix up the diskdef configuration. Perhaps in future I can also add a command to automatically generate a diskdef from a good raw dump, but that doesn't exist right now.

[crestr1]

Your initial advise was along the lines of yesterdays: "In this case the correct diskdef would be the standard single-sided dec.rx01 as shipped with Greaseweazle, except with heads = 2." It did not work and you sent me a suggested fix that added a side1 tail to the diskdef to try, this made the only the genuine ibm disks recoverable when we gave it start sector number of 27, for the other disks it still failed. Here the long nightmare started.

What was also missing was an --- " tracks 0-76. ibm.fm" apply to any heads

This diskdef seems to work properly: disk mot.mdos3 cyls = 77 heads = 2 tracks 0-76.* ibm.fm secs = 26 bps = 128 gap3 = 26 rate = 250 rpm = 360 end end

Need do some error management tests as final tests

[keirf]

FYI You should also be able to simplify the tracks line to tracks * ibm.fm

Everything else looks good. You don't necessarily need to specify gap3 but the inherited value from the default dec.rx01 format looks good. If you omitted it, Greaseweazle would pick a sensible value itself when writing back to disk.