Clock and data separation function.

[OSItester]

I am using flash floppy on an Ohio Scientific computer. It works well for both 5 1/4" and 8" drives. The only issue I have is that it requires an external clock/data separator to be inserted between the Gotek and the disk controller.

Would it be possible to have this function as a configurable option in Flashfloppy? Of course the timing would be different for different size disks.

A lot of eight inch drives have a separator already built into the drive. Most 5 1/4" drives don't have this built into them. The one used by Ohio Scientific for their smaller systems was made by MICRO PERIPHERAlS INC. Flexible Disk Series B51/B52. This drive used an optional built in clock/data separator. It used pin 30 for the separated clock and pin 34 for the separated data outputs.

See PDF page 12 for details of pin out. Document page 20. https://osiweb.org/osiforum/download/file.php?id=670

Doesn't show data separator. https://osiweb.org/manuals/MPI_B51-B52_Product_Manual_Long.pdf

BillDrom

[keirf]

Pretty sure someone did a patch for this. I'll see if I can dig it out.

[keirf]

There is code linked in issue #291 it needs cleaning up, being made configurable, and more testing.

[keirf]

Also I'm not 100% sure the separated clock/data signals here are actually the same as RDATA and read-window signals in #291. No doubt related, but might be different ways to the same end.

[OSItester]

The window you mention sounds like the window used in the external clock/data separator I use with the gotek. For 5 1/4" drives it is set to six microseconds and for eight inch drives it is set to somewhere between 2.5 to 3.0 microseconds. Depending on where you look. The one I use is basically just a 74121 monostable multivibrator with a potentiometer to set the pulse width and is triggered by the floppy read data. This pulse, one normal and one inverted are used to gate the read data using a 7400 IC producing the regenerated separated clock and the regenerated separated data. https://1drv.ms/b/s!Aipnd2AbiZTSiXxyi9uTdDiXdBEk?e=nFH93A

I am not a programmer so most of this is beyond me, but it seems that the output data could be ANDed with an internally generated pulse and output as clock and data. The generated pulse would probably have to be compared to the HFE file somehow to see if the data rate is for 5 1/4" or 8" drives in order to select the correct pulse width.

Another one I've seen uses a 74121 and a 7474 circuit but still has a pot to adjust. See page 16 of this PDF. https://osiweb.org/Peek65/Peek65_V4N5-05_1983.pdf

General info http://bitsavers.org/components/standardMicrosystems/TN6-1_9216_Floppy_Disk_Data_Separator_Jun82.pdf

Thanks again for all your efforts on this project.

[OSItester]

After reading issue #291, I looked at the NEC schematics linked there and the floppy drive interface appears to be using all odd number pins vice the normal even number pins. I don't know what the drive type is. It doesn't give the drive type number. A google search shows that the NEC FD-1155 is used on some NEC APC computers. I downloaded the drive manual for that eight inch drive and it uses all even number outputs like a more normal drive. It does have an output called Window on pin 50 and an output called MFM on pin 48 with regular read output on pin 46. On a Shugart 851 these pins are 50, separated clock, 48 separated data and 46 normal read output.

So, it would seem that Window is equivalent to separated clock and MFM is equivalent to separated data. On the NEC drive those output are only used if the VFO mod is installed. The schematics don't show that mod so I can only guess its a clock data separator like in the Shugart drive.

Jboone said he had his mod working but it sounds like he was using Window and normal read data, not MFM data. He seemed to also say he had issues keeping it synchronized. I would try it but don't know how to load his changes to the gotek.

It would seem that separated clock and separated data would always be in sync because it is just gated normal read data using the same normal and inverted gating signal. I'm guessing that the best configuration would require two pins on the gotek having to change. I don't know if pin 30 could be redefined as separated clock and pin 34 as separated data or if that is unreasonable. But it would make it exactly match the real 5 1/4" MPI floppy drive. Of course the ribbon cable could be modified to use other pins if necessary.

Thanks again for your work.

[OSItester]

After further study, I now know that the MFM signal is not separated data. It is a status signal.

It turns out that the FD-1165 drive switches the pin 46 read data output from normal read data to "standardized read data" when the VFO option is selected. I could not find a schematic for the VFO option board. This standardized read data is probably the same as so called separated data in the Shugart drive. The window signal is likely the same as separated clock in the Shugart drive.

In any event, both signals are required for proper decoding.

[keirf]

It sure would be easier if these signals were defined, or examples given, or someone could gather traces on a logic analyser!

[OSItester]

I don't have a logic analyzer, but I have scope images of the raw data, the separated clock and the separated data. https://imgur.com/a/RdXJqYH

The better separator circuits use a phase locked loop to get the synchronized clock but the Shugart and Siemens 8" drives just run the raw data into a one shot multi-vibrator to generate a 2.8 microsecond pulse at the trailing edge of the clock/data. They then use this pulse to gate the raw data/ clock input to select the data bits and output it as separated data. Any pulse that's not in the 2.8 uSec widow is a clock pulse and is output as Separated clock. At least that's how I understand it.

The Ohio Scientific disk controller writes each byte of data as a serial stream of 11 bits. One start bit , eight bits of data with the LSB first and one even parity bit and one stop bit. That is what is displayed on the scope traces

The data display shows the byte 18 from the disk. The test disk I used has 3600 bytes of 18 on every track. I included a partial schematic diagram of the separator circuit and a photo of the circuit board for good measure.

I hope this is of some help.

[snhirsch]

I'm in the same position and need separated clock and data for a Heath H-27 drive subsystem (used with their PDP-11/03 clone H-11). I do have a logic analyzer and can provide any required measurements. My system uses a WDC 1771 FDC controller rather than a UART if that makes a difference.

[keirf]

I'm in the same position and need separated clock and data for a Heath H-27 drive subsystem (used with their PDP-11/03 clone H-11). I do have a logic analyzer and can provide any required measurements. My system uses a WDC 1771 FDC controller rather than a UART if that makes a difference.

1771 looks more like raw data + read window (as described in #291)? That is no separation, but a square wave window signal?

[snhirsch]

I'm not intimately familiar with the low level details, but most (if not all) 8" drives from this period had "separated" outputs that should be well documented. If you have the time and interest in tackling this let me know what you need in terms of documentation and/or measurement?

[keirf]

It's not too hard to do as a build option for IMG image files. It's probably hackable for HFE image files too, but will need digging around for address marks to sync to clock vs data. Do you use IMG files?

[keirf]

Is the clock/data separation feature specific to FM (single density) recording?

[snhirsch]

Hi, Keir. I can use either raw sector image or HFE. If raw images are simpler to manage I don't think limiting to this format would be fatal. I believe clock/data separation was particular to IBM 3740 (FM) format and can find no mention of it being employed for MFM. Indeed, I saw language in one of the manuals to the effect that MFM and M2FM separation was always expected to be handled externally. Here are some extracts from the Shugart manusls. shugart_801.pdf shugart_850_851.pdf

Let me know if you need anything further?

[keirf]

Ok, thinking things through a bit, the separated data signal may have long periods with no pulse (eg, long sector data, all zeroes). The 16-bit timer @72MHz currently used to generate the RDATA output signal will overflow in <1ms. We ought to support 10s of milliseconds...

A few options:

1. Scale the timer rate down to say 1MHz. Fairly rational, not sure how wide is the code impact as a lot of assumptions about 72MHz rate are baked in here and there.
2. Chain two 16-bit timers. Never done this before. Probably more of a pain than (1).
3. Use a 32-bit timer. This is available only on the "new" Artery-based Goteks. This is very simple and therefore quite attractive. Is it reasonable to assume a new Gotek?

EDIT: (3) would require separated data on pin 34 and separated clock on pin 30, since the available 32-bit timer is on pin 34. Possibly it doesn't matter either way since a modified cable/adapter is required to interface to the host.

[keirf]

Reading the Shugart datasheets, FM data separation seems to often go in hand with hard sectoring. Is that a common pairing? I see for example that SA801 is for hard sectored disks, but SA800-1 is soft+separation.

Do you have any experience of this @ejona86 ?

[snhirsch]

Won't scaling down the timer frequency make things too granular and reduce timing accuracy? But, I am far from knowledgeable about FF internals so just thinking out loud.

In terms of options, I could live with (3). This is such a specialized application that buying a newer Gotek is a reasonable answer - to me, at any rate.

Finally, I have never seen nor owned a system that used 8" hard sector disks. I'm looking to replace the unreliable soft sector Memorex drives in my Heath H27 with Gotek based emulation. Others will have to weigh in on this. BUT, I would really like to see FF support hard-sector emulation, since I own Heath and Northstar systems that use 5.25" hard-sector diskettes (FM and MFM respectively, but not separated output).

[keirf]

This will probably be a special build or maybe even branch of FlashFloppy. All new Goteks I think are now Artery MCU so they're easy enough to find. We'll probably go with (3) then. I guess the original drive would normally emit raw read stream as well as separated streams: FlashFloppy won't do that, but I don't suppose it matters? Also it would be hardcoded in the build that D+C are always separated. For systems that need this, that would be expected and not a problem?

Regarding hard sector, I'm not clear whether HS is useful to you or not. However, HS work has been done in the v4 branch of FlashFloppy and was tracked in #400. Although that ticket is now closed you can feel free to post to it and @ejona86 would see it and help you try it out on your Heath and Northstar systems I am sure.

[snhirsch]

I had expected this to be a special mode of some sort and - no - it won't matter if that mode provides only separated output. Hard-sector is useful to me for 5.25" formats that have conventional combined clock & data signaling. Sorry for the confusion!

I didn't know that anyone had done work on hard sector emulation and will look at that issue. Thanks!

[ejona86]

I'm not familiar with this type of hard-sectored system, but the idea isn't strange given the mostly-independent clock separation logic used in FM systems.

I will note that (1) doesn't sound bad to me as only a handful of places touch the DMA timings and those same places will need to be audited for (3). But it actually seems like those places are already 32-bit safe, so the only real cost would be memory usage for the larger buffer.

My biggest concern would be DMA underrun as normally RDATA buffer stores ~1ms and now the code may need to fill 10x that before the buffer drains. It seems this shouldn't be a problem, but if something goes wrong I'd bet on that.

[keirf]

I will note that (1) doesn't sound bad to me as only a handful of places touch the DMA timings and those same places will need to be audited for (3). But it actually seems like those places are already 32-bit safe, so the only real cost would be memory usage for the larger buffer.

(1) may be worth a punt it's true. Prescaling the timer to 1x or 2x the bitcell timings is not something I've tried before.

My biggest concern would be DMA underrun as normally RDATA buffer stores ~1ms and now the code may need to fill 10x that before the buffer drains. It seems this shouldn't be a problem, but if something goes wrong I'd bet on that.

I think I see... well, let's say we have a really bad worst case and we have oceans of consecutive 0 data bits. Are you saying that we would accumulate into the next DMA sample until we see a 1 bit, but meanwhile the timer DMA catches up to this "sample in hand" and thus underruns? That's a tricky one, that is!

And for example, we may be held up in further reading from the image stream because our clock-bits and clock-DMA rings are full. Which may prevent us reading far enough to find the next D=1 bit.

I wonder how theoretical this all is. What's the real worst case? It seems to me most 8-inch FM formats are 128-byte sectors. Is that universally true? Then the worst case zeroes sequence would be 128byte16bits/byte2us/bit = 8ms ish.

Alternatively I may need to be able to pause the data timer, or re-stuff it with a new larger value. I don't know how plausible that is. A solution to this (re-stuffing or pausing the timer) might also solve the original problem and would not need 32-bit or prescaled timer.

What a pain!

[ejona86]

Yeah, that was my concern.

we may be held up in further reading from the image stream because our clock-bits and clock-DMA rings are full.

Oh, I hadn't considered having a "real" clock-DMA ring. Yeah, that would definitely cause trouble. My mental-model had assumed a looping circular buffer, like with a single entry, since the clock is steady. (Alternatively, using RAR or just leaving ARR in place and not updating it.) Synchronizing with data could be annoying, but I'd probably inject skew time into the first data pulse to get them aligned.

[keirf]

Data timings usually jitter a little to fit some imperfect number of bitcells into a perfectly-timed revolution. Which would cause a fixed clock to lose track. Of course we could Not Do That Then, and that could also make option (1) above more practical possibly.

Oh, actually also the Clock is not entirely steady, as Clock bits are dropped in sync marks. So there's that!

[ejona86]

I was really wondering about sync marks, because missing clock bits seemed out-of-place engineering-wise with pre-split clock.

SA800/801/851 have the separation circuitry (850 did not), where the 801/851 are for hard sectored media. The hard sector IBM format has no missing clock bits, so is trivial. Seems this includes IBM 3740. I'm weak at circuit-reading, but the early SA800 (see page 26 from SA800/801 Theory of Operation P/N 50664-0) and the Siemens FDD-100-8 both seem to lose clock sync during a single missing clock bit, such that they go out-of-phase and consider data 1 bits to be clock bits. However, they go back in-phase during/after the sync mark because of a data 0 bit. The SA800 had a later revision using VLI and the manual mentions this issue explicitly and it has a trace TS that can be used to give "true separation" that stays in-phase during sync marks vs "standard false separation" (see page 4-14 from SA800/801 Service Manual P/N 39025-1).

So the question is, if the sync marks mess up the FM clock sync does the controller make use of/correct for this or does the controller always emit clock bits, even if it is using soft sectors?

For the Ohio Scientific, the OSI Gazette in January, 1982 issue 20 (found via HxC forums) documents no special clock bits. But it also never mentions "FM." So maybe it isn't precise enough.

But Mark Csele quotes from Aardvark Journal, Feb. 1982 which confirms clock bits are always used:

After much searching and head scratching I concluded that since the OSI system never misses inserting the clock pulse before each data pulse that is written on the disk, the Data Separator would be easy and simple to design.

So clock/data separation seems largely for FM systems that never omit clock bits, as otherwise it was broken. There is some risk that a system is using a VLI SA800 with trace TS or similar equivalent circuit, but it may be fair to not worry about that case.

[ejona86]

Data timings usually jitter a little to fit some imperfect number of bitcells into a perfectly-timed revolution.

That is cyclical in nature, so we could just compute it once for the clock pulses and have it loop. There's probably no need to even sub-cycle-align the data and clock.

[keirf]

Regarding the scenario here of soft sector format with C+D separation (eg Ohio Scientific) I think we need a logic analyser trace from a real drive. We could then pull that apart to find the sector headers and work out what exactly is supplied by the drive when it sees a sync mark.

[geowar1]

I see SMC FDC9216B Floppy Disk Data Separators on eBay for ≈$7. IIRC it just needs an x4 clock. (So 500 kHz for 5”; 1 MHz for 8”?) It’s only 8 pins; it could be built up right on top of the A13 board.