search

davidgiven / fluxengine

PSOC5 floppy disk imaging interface
MIT License
357 stars 69 forks source link

Reading with `ibm` format is broken #606

Open tdaede opened 2 years ago

tdaede commented 2 years ago 
Error: LAYOUT: no sectors in sector definition!
Waffle2 commented 2 years ago

Agreed. Looks like the rest of the ibm formats got a new layout, but the ibm format did not. This impacts the ability to read trs-80 disks too. I have a single-sided, single-density example scp file, if you need it.

davidgiven commented 2 years ago

Oops, yes --- forgot to reply to this. This is actually intentional; originally the ibm format was supposed to autodetect the disk format, but in real life this doesn't actually work because it's far too easy to confuse the autodetection logic (such as by formatting a double-sided disk and then reformatting it as single-sided). So, this is now write-only and is intended for use with self-describing image formats only. I need to remove the imagereader declaration from this format and produce a better error message.

For reading and writing disks, you're now supposed to explicitly tell the system what format you're actually using. If there isn't one, then that is definitely a bug.

Waffle2 commented 2 years ago

The documentation (disk-trs80.md) suggests using

fluxengine read ibm -o trs80.jv3

What format would you suggest, for the current version?

davidgiven commented 2 years ago

There should be a matching ibm* format --- e.g. ibm720 for DSDD. The Tandy-ness should be autodetected. That should do for reads, but there needs to be a set of matching trs80 formats because special behaviour will be needed for writing due to the directory track.

Waffle2 commented 2 years ago

The disks I'm having trouble with are mostly "model 1" disks - that is single-sided, FM, 35 tracks. The ibm180 format (closest to the same) fails in three ways:

fluxengine read ibm180 -s newdosplus.scp -c 0-34 -h 0 -o newdosplus.dsk

This does a strange detection on track 1 (missing most of the track), and fails on sector 0 on track 17 (but reads the rest correctly). Last, it creates a 80 track double sided .DSK image.

fluxengine read ibm720 -s newdosplus.scp -c 0-34 -h 0 -o newdosplus.dsk

does better, correctly reading track 1, but again fails to read sector 0 of track 17, and still creates an 80 track double sided image.

Further, it would appear that imagewriter.cc no longer supports the creation of jv3 images thus why I'm using dsk in the examples here.

I have the scp file, if it is interesting...

davidgiven commented 2 years ago

Does the JV3 file extension no longer work? I did clean out some image formats which were no longer needed because ImgImageWriter supported them, but I thought I'd added extension support so nobody would notice...

Okay, this needs attention. Could you attach the SCP file and I'll take a look, please?

Waffle2 commented 2 years ago

On Oct 13, 2022, at 6:44 AM, David Given @.***> wrote:

Does the JV3 file extension no longer work? I did clean out some image formats which were no longer needed because ImgImageWriter supported them, but I thought I'd added extension support so nobody would notice...

Okay, this needs attention. Could you attach the SCP file and I'll take a look, please?

Sure

tdaede commented 2 years ago

While I would love to have the format ahead of time, for PC-88 and similar there are many common layouts, so the layout cannot be predicted ahead of time. Generating a layout based on what is seen on the disk is still important in this case.

On Thu, Oct 13, 2022, 6:46 AM Waffle2 @.***> wrote:

On Oct 13, 2022, at 6:44 AM, David Given @.***> wrote:

Does the JV3 file extension no longer work? I did clean out some image formats which were no longer needed because ImgImageWriter supported them, but I thought I'd added extension support so nobody would notice...

Okay, this needs attention. Could you attach the SCP file and I'll take a look, please?

Sure

Https://www.ainokea001.com/newdosplus.zip

— Reply to this email directly, view it on GitHub https://github.com/davidgiven/fluxengine/issues/606#issuecomment-1277641902, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ7NGWO2RY6L33F2FQY4B3WDAHC7ANCNFSM6AAAAAAQ2W4NX4 . You are receiving this because you authored the thread.Message ID: @.***>

davidgiven commented 2 years ago

The reason why that disk is failing to read data on track 17 is because there isn't any --- that track just contains noise!

davidgiven commented 2 years ago

I've added TRS-80 formats that should read SD and DD disks, 35 and 40 tracks, to the trs80 branch. PTAL? 35 track is the default; for 40 track, add --40 to the command line.

I managed to make them round-trip in the tests but I doubt they work for writing. I need more information about the exact encoding of the various non-standard TRS-80 IDAM bytes, and it'd be really useful to have a volunteer with an actual TRS-80 I or II (or III or IV) who can test them.

Waffle2 commented 2 years ago

On Oct 13, 2022, at 1:56 PM, David Given @.***> wrote:

I've added TRS-80 formats that should read SD and DD disks, 35 and 40 tracks, to the trs80 branch. PTAL? 35 track is the default; for 40 track, add --40 to the command line.

I managed to make them round-trip in the tests but I doubt they work for writing. I need more information about the exact encoding of the various non-standard TRS-80 IDAM bytes, and it'd be really useful to have a volunteer with an actual TRS-80 I or II (or III or IV) who can test them.

Apparently, the directory of that original newdosplus disk got clobbered at some point.

Here is another image that I’m able to extract clean: https://www.ainokea001.com/newdos80.zip https://www.ainokea001.com/newdos80.zip

This was produced from my original newdos80 disk. I do have a model I, and a 4p. The model I is not set up, but I will do so.

Thank you for the changes! Tim Mann has some good write ups on the formats:

https://www.tim-mann.org/trs80/dskspec.html https://www.tim-mann.org/trs80/dskspec.html https://www.tim-mann.org/trs80faq.html#[12] https://www.tim-mann.org/trs80faq.html#%5B12%5D

Waffle2 commented 2 years ago

On Oct 13, 2022, at 1:56 PM, David Given @.***> wrote:

I've added TRS-80 formats that should read SD and DD disks, 35 and 40 tracks, to the trs80 branch. PTAL? 35 track is the default; for 40 track, add --40 to the command line.

I managed to make them round-trip in the tests but I doubt they work for writing. I need more information about the exact encoding of the various non-standard TRS-80 IDAM bytes, and it'd be really useful to have a volunteer with an actual TRS-80 I or II (or III or IV) who can test them.

Good news - this does correctly read the newdos80 scp image, and will create a dsk file. I was not able to use the command line version on my 40 track drive, but it was able to read correctly from my HD drive.

The bad news - it still cannot seem to create jv3 files, and the dsk and other formats do not carry the DAM information ,so while the imaged disk is able to boot (kind of), it fails to load completely because it can’t find the directory. Is there a supported format that carries the address mark info?

davidgiven commented 2 years ago

That's good news.

Your 40-track problem seems to be caused by the SCP file not having the correct header bits set. It claims to have been read on a 96tpi drive, even though it only contains tracks 0..34. What are you using to create it? (By which I mean --- was it FluxEngine, meaning there's something else I need to fix?)

Re the JV3 file format: I don't believe it ever did support writing JV3 files. (It supports reading them.) I don't think there is a format which supports DAM markers. It's probably easiest to fake, by writing a JV3 writer which always sets the special DAM for every sector in track 17.

Regarding real hardware: it'd be really helpful if you could format blank disks for as many TRS-80 formats as you can manage and then send me the flux files for inclusion in the https://github.com/davidgiven/fluxengine-testdata repository. They'd be invaluable for testing. Apart from anything else I can examine them to find out how they work regarding timing etc.

You might have luck writing DD disks (512-byte sectors) with FluxEngine and then using them in the real hardware, but I doubt SD disks (256-byte sectors) will work at all right now.

Waffle2 commented 2 years ago

On Oct 13, 2022, at 3:48 PM, David Given @.***> wrote:

That's good news.

Your 40-track problem seems to be caused by the SCP file not having the correct header bits set. It claims to have been read on a 96tpi drive, even though it only contains tracks 0..34. What are you using to create it? (By which I mean --- was it FluxEngine, meaning there's something else I need to fix?)

I used gw directly.

gw read --tracks "c=0-34:h=0" --drive 1 foo.scp

Drive 1 is my 40 track drive. Drive 0 is my HD drive.

Re the JV3 file format: I don't believe it ever did support writing JV3 files. (It supports reading them.) I don't think there is a format which supports DAM markers. It's probably easiest to fake, by writing a JV3 writer which always sets the special DAM for every sector in track 17.

My confusion then - I was basing this on your documentation “disk-trs80.md”, which says:

fluxengine read ibm -o trs80.jv3

So I assumed it supported writing…. No worries. I’ll cobble something up in perl :-)