Micropolis disk captured as .scp on greaseweazle - any hope?

[cube1us]

I captured a .scp image using a greaseweazle (V0.27) using this command using a Micropolis 1042 mod I (SSDD, 35 tracks, 143KB total capacity)

gw.exe read --drive 0 --rpm 300 --rate 250 --tracks c=0-34:h=0 --revs 60 disk38.scp 1>disk38.log 2>&1

I then tried to run it through fluxengine thusly and with these results - no .img file was created:

fluxengine read config.micropModI -s disk38.scp -o disk38.img SCP tracks 0-34, heads 0-0 SCP sample resolution: 25 ns 0.0: Error: SCP read I/O error: No error <<<<<<<<<<<<<<

Did I miss something? If not, any hope of this working, or is it a dead end (in which case I can certainly acquire the necessary hardware to use fluxengine directly.)

The profile I used (config.micropModI) was as follows:

decoder { micropolis { } } cylinders { start: 0 end: 34 } heads { start: 0 end: 0 } comment: "Micropolis Mod I MetaFloppy 143KB 5.25\" SSDD hard-sectored (ro)" image_writer { filename: "micropolis.img" img { } } flux_source { drive { drive: 0 hard_sector_count: 16 } } flux_sink { drive { drive: 0 } }

[davidgiven]

That looks very much like a bug at my end, judging by the error. Is it possible to attach the SCP file? One thing you could also try is using the FluxEngine client to read directly from the GreaseWeazle --- you will need to tell it which tty port it's on but it should work fine.

[cube1us]

Reading hard sector using the fluxengine client via gw hardware is not supported - or so the error message it gave told me. I seem to have deleted the Mod I (disk 38 scp), but here is a Mod II .scp (disk50 - a bit more than twice as many tracks - but gave the same error message), along with the gw capture log. The profile used for this one was:

decoder { micropolis { } } cylinders { start: 0 end: 76 } heads { start: 0 end: 1 } comment: "Micropolis MetaFloppy 630kB 5.25\" DSDD hard-sectored (ro)" image_writer { filename: "micropolis.img" img { } } flux_source { drive { drive: 0 hard_sector_count: 16 } } flux_sink { drive { drive: 0 } }

disk50.log disk50.scp.gz

[ejona86]

I know this is about SCP, which is not as easy for me to help with. And I see #328 which leads me to believe you got some strategy working (edit: it seems you got a fluxengine). But I did want to mention:

Reading hard sector using the fluxengine client via gw hardware is not supported - or so the error message it gave told me.

That is sorta the case, as the fluxengine firmware since #200 is able to do things like properly measure the RPM of a hard-sectored disk and read from the beginning if you request it. However, you can hack around lack of hard sector firmware support by just requesting lots of rotations like you did with gw. My greaseweazle reads micropolis just fine via --flux_source.drive.hard_sector_count=0 --flux_source.drive.revolutions=20. Although I'll note that sometimes the rotational speed is detected to be 6ms instead of 12ms, in which case I just re-run the command.

[ejona86]

I figured out the source of the error: fluxengine has a limit of 5 revolutions for scp, which wasn't validated. This led to reading uninitialized data and brokenness. Hacking to increase the revolutions lets things get much further, but there's still breakage that needs to be investigated.

[ejona86]

Tracked down the second issue to null pointer. I'll replace the revolution hack with something more rigorous and then I'll send out a PR.

diff --git a/lib/fluxsource/scpfluxsource.cc b/lib/fluxsource/scpfluxsource.cc
index 97499ce..228b80a 100644
--- a/lib/fluxsource/scpfluxsource.cc
+++ b/lib/fluxsource/scpfluxsource.cc
@@ -59,7 +59,7 @@ public:
                int strack = strackno(track, side);
                uint32_t offset = Bytes(_header.track[strack], 4).reader().read_le32();
                if (offset == 0)
-                       return std::unique_ptr<Fluxmap>();
+                       return std::make_unique<Fluxmap>();

                ScpTrack trackheader;
                _if.seekg(offset, std::ios::beg);
diff --git a/lib/scp.h b/lib/scp.h
index 7df3be3..9f8dc92 100644
--- a/lib/scp.h
+++ b/lib/scp.h
@@ -37,7 +37,7 @@ struct ScpTrack
         uint8_t length[4]; // number of bitcells
         uint8_t offset[4]; // offset to bitcell data, relative to track header
     }
-    revolution[5];
+    revolution[50];
 };

 #endif

./fluxengine read micropolis -s disk50.scp -c 0-34 -h 0
SCP tracks 0-76, heads 0-0
SCP sample resolution: 25 ns
  0.0: 7996.274166666666 ms in 1569971 bytes
       36 records, 36 sectors; 34.65us clock (29kHz); 16 distinct sectors; 
       logical track 0.0; 9216 bytes decoded.
...
     Tracks -> 1         2         3         
H.SS 01234567890123456789012345678901234
0. 0 ...................................
0. 1 ...................................
0. 2 ...................................
0. 3 ...................................
0. 4 ...................................
0. 5 ...................................
0. 6 ...................................
0. 7 ...................................
0. 8 ...................................
0. 9 ...................................
0.10 .......................X...........
0.11 ......................X............
0.12 .......................X...........
0.13 ...................................
0.14 ...................................
0.15 ...................................
Good sectors: 557/560 (99%)
Missing sectors: 3/560 (0%)
Bad sectors: 0/560 (0%)
IMG: wrote 35 tracks, 1 sides, 140 kB total

[davidgiven]

Sorry for the delay; been on holiday...

Thank you so much for tracking this down. Interestingly, I was talking to the scp author and they said there was a limit of five revolutions, which is why the structure was set like that.

[ejona86]

It does seem 5 revolutions is the norm, but the current spec text seems to have no limits. I think it made a lot of sense to limit the implementation to 5 at the time, although it would have been good to validate the limit. But this issue did let me discover that SCP is actually useful for hard sectors, which is nice!

From https://www.cbmstuff.com/downloads/scp/scp_image_specs.txt:

BYTE 0x05 is the number of revolutions, which is how many revolutions for each track is contained in the image.

And:

Starting at BYTE 0x04 are three longwords for each revolution that is stored. Typically, a maximum of five sets of three longwords are stored using the SuperCard Pro's imaging program, but this is user defined and can vary from image to image. Using BYTE 0x05 of the main file header, you can determine the number of sets of three longwords (one for each revolution stored).

It says "typically" but no hard limits. There's quite a few places in the text it gives typical values (for other things as well) but doesn't actually limit what files might use. Contrast that with the number of tracks, which has a clear limit:

BYTES 0x06 and 0x07 are the start track and end track bytes. Tracks are numbered 0-167, which is a maximum of 168 tracks (84 tracks with top/bottom).

And that limit is repeated elsewhere:

BYTE 0x03 contains the track number (0-167).