EFM filter/PLL in ld-decode requires tuning

[simoninns]

From discord:

7:19 PM] simon_dd86: on another note @happycube and ats - I've been looking into the errors around the EFM decoding... one conclusion is that the sync detection before EFM decoding (i.e. sync in the raw t-values) has way too many errors [7:19 PM] simon_dd86: which points towards the filter/PLL as a big contributor to the issues [7:19 PM] DdD_irc_discordbot: Yes, that seems likely [7:19 PM] simon_dd86: (it shows up as sync over/under shoot in the logs) [7:20 PM] DdD_irc_discordbot: I've seen dramatically different EFM performance between different captures of the same (skippy) disc when doing Lain recently, so I expect there are more problems where the PLL doesn't lock very well [7:21 PM] happycube: that makes sense - gonna be a pain to debug

DdD_irc_discordbot: It could really use some attention from someone who's good at digital PLL design (i.e. not me!) [7:24 PM] simon_dd86: @happycube - I was looking into the debug issue... but there is a way I think I've found... [7:25 PM] simon_dd86: each channel frame is 588 bits; and each frame is sync'd with 100000000001000000000010 - so you could use the sync pattern and known gap to see if the PLL is outputing nonsense

[7:25 PM] simon_dd86: ECMA-130 pdf page 27of57 (spec page 17)

[7:26 PM] simon_dd86: you could hook some code up to the PLL output as a "good/bad" light and then swing the values to see what works

[atsampson]

"An All-Digital Bit Detector for CD Players" (unfortunately paywalled) is the best description I've seen of how EFM filters/PLLs worked in the early 90s - it describes several things that the current PLL doesn't do, e.g. correcting for underexposed/overexposed discs, and using a separate frequency loop to pull in faster.

If anyone's playing with the filter, edit-efm-filter may be useful to you. The current filter was tweaked by hand so I'm sure there's scope for improvement.

[blindrezo]

Hey guys,

I ran into issues with 2 copies of the same disc and made sample clips of each, at the same location.

This was captured from a calibrated CLD-S315. My computer is a Ryzen 1700, Crosshair VI Hero x370, 16GB RAM.

I hope that I did this right and that this is helpful. :)

The samples, along with the EFM decode debug info can be found in the link below:

https://drive.google.com/drive/folders/1tnCq7SikAJDrXop-NeCB-7ZQVxsJrTN-?usp=sharing

Thanks.

[simoninns]

Trying to get some clues on this one... I had the idea of sampling the EFM direct from a LD-V4300D's EFM output DIN connector. Looks to me like the output is a lot less asymmetric than the ld-decode output and also has less ISI (if that's what causes the dipping in the longer half-waves of ld-decode's output). Attached is a PDF of the oscilloscope traces (from the Roger Rabbit PAL bonus disc:

Roger_Rabbit_PAL_EFM.pdf

[simoninns]

This is the output from ld-decode (same disc as above, RR PAL bonus). There is far more distortion and the waves are highly asymmetric (which is what I think is really throwing the ZC detection off at the moment). This is all pre-PLL, so is really the ground work of improvement.

[atsampson]

Yes, that looks like the filter shape isn't very well matched to that disc. From when I was playing with the filter before, the asymmetric peaks result from phase errors - and it looks like the LF amplitude isn't right either - the amplitude shouldn't vary much over time.

Just out of interest, have you tried capturing the EFM output using a DDD? It'd be interesting to see what the DDD's filter does to the waveform (if anything), compared to the flat version from your scope...

[simoninns]

It would take some work to get it all set up to test the DdD against the EFM output; I can do that, but I'm not sure I follow the reasoning... Since the EFM (in the LD) is modulated into the overall signal (rather than being the only signal as with CDs) - I can't see how the DdD could affect it (since it is within the expected spectrum and FM). Just reply "DO IT" and I will though :)

(edit: I should mention that the Roger Rabbit bonus disc has no issues with EFM decoding, the audio is recovered. I picked it since it was a known-good rather than a known bad - so we're looking at near-best-case).

[simoninns]

Related to issues #574 #523 and issue #528

[staffanu]

I built a replica of the EFM filter in the LD-V4300D in order to use to for EFM output for another player (pre digital sound era). Just for comparison with the bode plot at https://www.domesday86.com/?page_id=2678, which I assume is from simulation of the filter, here is a bode plot (and csv data) for the filter that I built.

bedeplot-ld-v4300d.xlsx

[happycube]

@staffanu - forgot to take a look at this, I really should give it a spin. Quite nice that you built a replica!

OTOH, I did some more tuning of the existing EFM filter and it reduces C1 errors on a lot of disks. It's quite possible a more exact replication would do even better...

[staffanu]

I built this since I was working on a hardware (FPGA) EFM decoder. I still am, but had some other projects usnig up alot of time lately, so actually, by coincidence I was looking at the EFM code again for the first time in about half a year. Anyway, I never actually tried the filter since I decided to first make the decoder work on the EFM signal output from my LD-V8000. (since that is a signal that I actually know can be decoded). The filter was mostly a fun thing to build, and I'm still thinking it will be useful once I have the decoder working. That being said, I own a Pioneer PD-X707L CD player, which has an EFM input. I used it to make sure the output from the V8000 is ok, but it never really occurred to me until now to add the filter to the Domesday86 output signal, and then run that through the CD player. It will be a fun exercise in the near future!

[atsampson]

I think we can probably say this is fixed since c2dbf40d28dca339922cf7716df8d277c72656b8 - I suspect further improvement would need an adaptive filter, which is a different problem.

[atsampson]

Just out of interest, have you tried capturing the EFM output using a DDD?

It's only taken two and a half years, but I did finally get around to building an adapter and giving this a try. On EE 1077 side 1, the usual RF capture toolchain struggles badly:

Info: F1 Frames to Audio:
Info:        Audio samples: 120213324
Info:      Corrupt samples: 0
Info:      Missing samples: 9670836
Info:    Concealed samples: 33477462
Info:        Total samples: 163361622

Whereas feeding a DDD capture of the EFM output into the EFM PLL gives an essentially perfect result:

Info: F1 Frames to Audio:
Info:        Audio samples: 163632168
Info:      Corrupt samples: 0
Info:      Missing samples: 9408
Info:    Concealed samples: 0
Info:        Total samples: 163641576

A slightly rotten copy of EE 1035 side 4 is very poor with an RF capture:

Info: F1 Frames to Audio:
Info:        Audio samples: 10336038
Info:      Corrupt samples: 0
Info:      Missing samples: 67367748
Info:    Concealed samples: 67484586
Info:        Total samples: 145188372

But a lot better with an EFM capture:

Info: F1 Frames to Audio:
Info:        Audio samples: 163019874
Info:      Corrupt samples: 0
Info:      Missing samples: 412776
Info:    Concealed samples: 146598
Info:        Total samples: 163579248

So that confirms that the DDD filter, the PLL and ld-process-efm are all fine; it's definitely the EFM filter that isn't behaving.

(If anyone else wants to try this, you may find this branch that resurrects the C++ PLL, find-efm-start and decode-efm useful...)

[happycube]

Any chance you could do a similar comparison using a disk that ld-decode's current code can decode properly? It'd be interesting to see the difference in waveforms.

I looked at this a bit a few days ago, haven't cracked anything yet.

I was pondering if this was a DDD filter issue, but since you captured using that, it thankfully isn't. (whew!)

[happycube]

Here are the EFM filter circuits from the CLD-S180/S280(/S105) service manual. It's definitely a non-linear filter with a couple of comparators and filtering after those (i.e. a reshaped signal)

The first one is the circuit inside the PD0003 chip, which goes from pin 58 (LDD) or 57 (CD) to pin 52. (The triangles are comparator circuits which 'sharpen' the output)

And the second one shows the filters before that.

---

I'm not sure I can pull this one off, but I'll keep at it for a bit :)

@staffanu - got any code/SPICE stuff from that filter work last year you can (re?)share?

@atsampson - also can you do audio CD capture on your setup for comparison? IIRC you used a 4400 so probably not, but there'd be a good bit less filtering involved.

[happycube]

Interestingly, I just looked at a CDDA capture off my V2800/DDD, and it decoded an entire song mostly cleanly with the occasional pop/lost sample, without any filtering:

Info: F2 Frames to F1 Frames: Info: Valid F2 frames: 1664166 Info: Invalid F2 frames: 658 Info: Initial padding frames: 14406 Info: Missing section frames: 3430 Info: Encoder off frames: 196 Info: TOTAL frames: 1682660 Info: Info: Frames start time: 00:01.73 Info: Frames end time: 03:48.70 Info: Info: F1 Frames to Audio: Info: Audio samples: 9984996 Info: Corrupt samples: 0 Info: Missing samples: 20580 Info: Concealed samples: 3948 Info: Total samples: 10009524 `

[happycube]

About an hour ago, I got an .oga in my gdrive from Gamn (thanks!). Decoded a bit worse than my CDDA cap:

Info: F2 Frames to F1 Frames: Info: Valid F2 frames: 1614357 Info: Invalid F2 frames: 30181 Info: Initial padding frames: 26166 Info: Missing section frames: 23618 Info: Encoder off frames: 0 Info: TOTAL frames: 1694322 Info: Info: Frames start time: 00:03.43 Info: Frames end time: 03:50.39 Info: Info: F1 Frames to Audio: Info: Audio samples: 9686142 Info: Corrupt samples: 0 Info: Missing samples: 141708 Info: Concealed samples: 181086 Info: Total samples: 10008936 Info: Info: Audio start time: 00:03.43 Info: Audio current time: 03:50.39 Info: Audio duration: 03:46.71

[happycube]

Playing around with gain levels - the current test (w/o any RF filtering) is very sensitive to clipping, but even well below that it there are still errors.

That there are so few means a working cd-decode isn't as far off as I thought.

[staffanu]

I've seen the messages but haven't got around to actually do anything. Is there a set of test tracks that we could use to compare the performance when experimenting with filters? When I was more into efm decoding I had a few discs I knew were problematic, and they seemed pretty consistently the same over time (it could have been that changing filters etc made other discs difficult).

I've not done any modeling of the filter I built: I just copied Pioneer's design assuming it would be good enough. And it was: I successfully added digital audio output to my Philip VLP-600 using an fpga to process the signal after the filter. (All this from an idea after repairing it and realizing it only had analog audio -- how hard can it be to add a decoder? ;) )

Staffan

On Sat, Oct 14, 2023, 18:14 Chad Page @.***> wrote:

Playing around with gain levels - the current test (w/o any RF filtering) is very sensitive to clipping, but even well below that it there are still errors.

That there are so few means a working cd-decode isn't as far off as I thought.

— Reply to this email directly, view it on GitHub https://github.com/happycube/ld-decode/issues/528#issuecomment-1763028839, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABRHXJAAYH5MMJWILUEBQU3X7K255AVCNFSM4PZJF5TKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZWGMYDEOBYGM4Q . You are receiving this because you were mentioned.Message ID: @.***>