Hi8 and Video 8 not yet working properly in PAL

[davidvankemenade]

It seems vhs-decode doesn't yet work properly with Video8 and Hi8, at least in PAL. The video is noisy and there are horizontal lines in high-frequency areas. There's little chroma.

I get very similar results with different Sony models: TRV-240 camcorder, Sony EVO-9700 and Sony EVO-9800P recorders.

It seems this behaviour is virtually the same for Hi8 and Video8.

Sample frame below, sample capture file in this post: https://www.reddit.com/r/vhsdecode/comments/12ocbd0/hi8_capture_not_decoding/

PAL Hi8 frame, with --chroma-gain 4.0

[oyvindln]

Video8 and hi8 are not fully implemented yet, especially chroma. The chroma signal on 8mm has an emphasis filter applied unlike older formats, and both that and the luma emphasis filters are not straight forward linear filters but depend on both input amplitude and frequency, which is a process that I haven't found a lot of info on how to implement digitally so might take some time.

I haven't experienced black/white streaks to that extent on the samples we have so far though I'm not sure if I have any Hi8 PAL ones so even the basic rf filters to filter out the part of the signal we want might be off rather than anything with your setup. Though as noted even if that works it won't work correctly as of now.

EDIT: Streaks likely due to sample having been resampled to 16 mhz which isn't sufficient bandwidth for hi8.

[davidvankemenade]

Regarding the streaks (and excluding the chroma for now), I'm not sure what settings I used to create the compressed file, so you may well be right that it was resampled. In order to exclude compression issues, I've posted a raw 16-bit 10-second DdD capture using the same tape but a different recorder (Sony EVO-9800P): https://drive.google.com/file/d/1-cR_UXjno9nROC35q3RCSj4WDJF50dbw/view?usp=share_link As far as I can see at the moment, feeding this directly into vhs-decode still gives similar results with black/white streaks. I also get similar results when I play back a Video 8 tape in the same recorder. So the streaks remain a mystery for now. Or am I overlooking something?

The 8mm emphasis filters being non-linear and input-dependent sound very complicated. Can I help in any way by providing more samples, doing research or anything else? Which information around the format specifics are already known?

[oyvindln]

Looking at the frequency spectrum in that last one, the higher frequencies where the luma/brightness are look pretty attenuated compared to the chroma/audio part. Not sure what's causing this, haven't seen this on any other samples.

Your sample (use --dp demodblock as parameter to show this and other plots:

Different cap from the drive to illustrate (NTSC one, not sure if we have a PAL hi8, though the frequency bands used for NTSC are abou the same as for PAL.) Exact amplitudes won't be directly comparable since this is a 8-bit capture from a cx card.

[oyvindln]

I managed to get it a bit cleaner by messing with the rf filters though but I don't know what's causing the difference in frequency response between your cap setup and others. I think the code might need a stronger filter in any case to filter out more of the audio and chroma signal from the luma one even for the ohter samples.

[oyvindln]

Re specifications, the details are mostly available. There is a standards document IEC 60760 for the base formats, and schematics for the early models where it's done using mostly discrete components show the actual circuits (some of the Sanyo ones have a bit of detailed description of it too.) It's more the issue of how implement it digitally that's tricky as it's a very niche topic so would maybe be helpful to get some help from someone with very extensive DSP programming knowledge or something.

Samples are always useful of course.

[davidvankemenade]

Hi @oyvindln , thanks for looking into this last month. I've done a fresh test with Sony EVO-9800P and the result looks lot better. The horizontal streaks seem to have mostly gone and the chroma seems surprisingly good.

This is with default (1.0) chroma gain. The main change on my side is that I've soldered a BNC cable directly to my RF tap instead of using a BNC probe. This led to higher RF signal levels, so I've reduced the DdD gain to get a decent capture level.

The frequency spectrum through --dp demodblock now looks more like your earlier example:

Many scenes now look quite good. My main concern right now is that vertical lines are all over the place. I've got an extreme example here: https://drive.google.com/file/d/1jd2vxLL1DygdkBC1KsSCpIVDXDNDAMNk/view?usp=drive_link

The lines at 2.0, 1.8, 1.6 etc. should all be perfectly vertical and they are on the analogue playback. vhs-decode seems to struggle with this somehow and it's visible on the right-hand side of the frame.

Do you have any idea what's happening in this case?

[harrypm]

@davidvankemenade have you made a reference capture with conventional means to reference the decode?

[davidvankemenade]

Hi @harrypm, I've just made a reference capture: https://youtu.be/y3-ps2-eINM?t=236

[harrypm]

Can you put the file on the drive, if it's gone though YTs compression its not really a reference...

(Also YouTube doesn't support interlaced it butcheres it, QTGMC de-intelrace and upscale to 1440x1080 is the minimum if you want to show 4:3 analogue media on YT ideally with a visually lossless native codec like ProRes HQ etc even then it's still bias compressed on blacks on the image etc, hence why we use Odyssey for demo videos now while it's limited it can actually stream SD natively atleast)

---

From: davidvankemenade @.> Sent: Sunday, June 25, 2023 12:36:46 PM To: oyvindln/vhs-decode @.> Cc: Harry Munday @.>; Mention @.> Subject: Re: [oyvindln/vhs-decode] Hi8 and Video 8 not yet working properly in PAL (Issue #88)

Hi @harrypmhttps://github.com/harrypm, I've just made a reference capture: https://youtu.be/y3-ps2-eINM?t=236

— Reply to this email directly, view it on GitHubhttps://github.com/oyvindln/vhs-decode/issues/88#issuecomment-1606050747, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANOFJIACRDKCBJCASP74JQDXNAPE5ANCNFSM6AAAAAAX3VMQL4. You are receiving this because you were mentioned.Message ID: @.***>

[davidvankemenade]

I've uploaded an MKV here: https://drive.google.com/file/d/1EuGVn-3O--YSEdytOmmq4Gnf1k3WLsP4/view?usp=drive_link

The 20-section captured in the 16-bit flac file starts around 4 minutes.

[oyvindln]

Improved things a bit in 1ef6d3d68810ed139b4666ab00fd30d79522af53. The high pass filter on the rf signal was clearly set too up till now low so the AFM carriers were not getting filtered out before the luma was demodulated but now it should get rid of much more of them. Also got the deemphasis filter a bit closer to the correct one for hi8 which reduced smearing a little. Needs further work though, and not sure what's causing the tbc to freak out yet.

[davidvankemenade]

Thanks for your improvements @oyvindln!

Your changes prompted me to read up on the video standards and try to understand how various parameters, e.g. in video8.py, relate to them.

For example, I experimented with sharper cutoff outside the Hi8 side bands.

    # Sharper upper cutoff to get rid of high-frequency junk.
    rfparams["video_lpf_extra"] = 8810000
    rfparams["video_lpf_extra_order"] = 0 # DvK: was 3
    rfparams["video_hpf_extra"] = 2520000
    rfparams["video_hpf_extra_order"] = 0 # DvK: was 3

Before, the test card decoded like this:

When I change the order to 0, I get this:

There is more chroma noise as would be expected. The reduced horizontal black lines are an improvement though. On the downside, the vertical lines are more jagged and the TBC seems to struggle more.

The bit below seems to be aimed at reducing streaks, but it is disabled:

    # Filter to pull out high frequencies for high frequency boost
    # This should cover the area around reference white.
    # Used to reduce streaks due to amplitude loss on phase change around
    # sharp transitions.
    rfparams["boost_bpf_low"] = 7200000
    rfparams["boost_bpf_high"] = 7800000
    # Multiplier for the boosted signal to add in.
    rfparams["boost_bpf_mult"] = 0 # was 0

When I tried to put in a non-zero multiplier, the results got worse rather than better. Why would this be the case?

What is the rationale behind all the settings here? For example:

    rfparams["deemph_mid"] = 550000 # TODO: Not correct, needs to be fixed properly
    rfparams["deemph_gain"] = 14

Why this frequency and gain?

    # Parameters for high-pass filter used for non-linear deemphasis, these are
    # probably not correct.
    rfparams["nonlinear_highpass_freq"] = 600000
    rfparams["nonlinear_highpass_limit_h"] = 5000
    rfparams["nonlinear_highpass_limit_l"] = -20000

Where does the frequency come from? What do the limits mean and what is their dimension?

    # Video Y FM de-emphasis
    rfparams["deemph_tau"] = 0.47e-6 # From spec

Do you have a reference to the spec so that I can try to understand?

    # Video EQ after FM demod (PAL VHS)
    rfparams["video_eq"] = {
        "loband": {"corner": 2.62e6, "transition": 500e3, "order_limit": 20, "gain": 2},
    }

What is the rationale behind these parameters, applied to VHS to begin with?

[oyvindln]

For filter order - higher order, not lower, will give a sharper filter (though there are some tradeoffs, plenty of materials on digital filters on the web if you are curious.)

The boost bits an extra boost to the frequencies around the top of the luma frequency band used to compensate for the frequency response of the output from the head amp on vcrs though haven't tuned the params all that much for 8mm yet.

deemph_mid/gain is based on the linear part of the deemphasis filter mentioned in the standards documents for the hi8 format (not something I can link directly here.) Right now the values are a bit guesstimated extrapolated from how they are on u-matic. Need to load up the exact values in a circuit simulation or calculate the filter properly to get it more exact. (Still missing the non-linear bit though which will have some impact on the final result too.) Same with the "deemph_tau" value which is directly from the standard though the value isn't directly used atm.

nonlinear_highpass_x is used for the --nld option, it's not altered from the values used for vhs - it was a basic attempt at a non-linear response based on a patent I found by extracting the high frequency portion of the signal, limiting it and then subtracting it from the original to attenuate low amplitude high frequencies a bit.

video_eq is used for the --sharpness option, it's a basic eq on the video output to sharpen/soften the image, again not tweaked from the values used for VHS.

Materials around the 8mm format are a bit scant as it came out a bit later when circuitry had been minimized and sony wasn't as big on putting long explanationsin the ir service manuals but if you are curious about the workins of videotapes, this document about VHS is a useful start: https://freeservicemanuals.info/en/servicemanuals/viewmanual/JVC/VIDEOTECHNICALGUIDE/GENEARAL/ The general principles on how video is stored is the same between 8mm, vhs, betamax, u-matic and several others - 8mm adds some extra steps with chroma filtering and differs a lot from the earlier formats when it comes to tracking and audio is handled.