Fix sync level for real baseband file output

[Zcooger]

Using file output produces grayscale baseband set of raw images in desired type. I'd like to correct horizontal synchronization level of the output because it seems incorrect for digital representation of analog composite TV signal where sync pulse level in 8bit colorspace lies at 0 and backporch at around 60 ~64. At this moment I tried modifying the gamma value for real PAL mode but it gave worse result.

[fsphil]

hacktv's output levels are related to voltage. For an int16 file, each sample is the voltage multiplied by INT16_MAX. The voltage for PAL ranges from -0.3 to 0.7 (+some PAL subcarrier overshoot) so in the int16 file that is roughly -9830 to 22937.

I believe the standard you refer to is Rec.601 which specifies 8 and 10-bit levels at 13.5MHz: https://www.itu.int/rec/R-REC-BT.601-7-201103-I/en

The levels could be adjusted to match, but annoyingly it's going to be different for uint8 and uint16 modes. I'm not sure how best to do it.

[Zcooger]

I refered to values visible on our computer screens - original captures (from LaserDisc for example) converted to standard RGB bitmap have these or similar values. Maybe adjust it for uint16 for this moment.

[Zcooger]

Ok, so I know where to dig for testing. INT16: -9830 to 22937 UINT16: 22937 to 65535 (+32767) INT8: -76 to 89 UINT8: 51 to 178 (+128)

[Zcooger]

Annoyingly or not it may be dirty workaround - I think it may cause dithering but who will notice with naked eye?: For UINT16 = (INT16 + 9830) 1,4 For UINT8 = (INT8 + 77) 1,4

[neo7530]

@Zcooger - how did you open the raw-sample in ld-analyze? have tryed it, but run into troube because of missing json metadata... can you help me about that?

[neo7530]

btw, i tryed to convert the values manually by using this formula:

return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;

this does a mapping from 89d - 255d to 00d - 255d and worked fine on uint8 output.

[Zcooger]

@neo7530 you have to prepare such file with amount of frames generated by HackTV and descripted in this code (2 fields of 14MHz), if you want more you have to copy and paste manually:

{
    "pcmAudioParameters": {
        "bits": 16,
        "isLittleEndian": true,
        "isSigned": true,
        "sampleRate": 48000
    },
    "videoParameters": {
        "numberOfSequentialFields": 26,
        "isSourcePal": true,
        "fsc": 4433618.75,
        "fieldWidth": 896,
        "sampleRate": 14000000,
        "black16bIre": 16383.0,
        "white16bIre": 54611.0,
        "fieldHeight": 312.5,
        "colourBurstStart": 78.0,
        "colourBurstEnd": 110.0,
        "activeVideoStart": 146.0,
        "activeVideoEnd": 874.0
    },
    "fields": [
        {
            "isFirstField": true,
            "syncConf": 100,
            "seqNo": 1,
            "audioSamples": 0,
            "diskLoc": 0.0,
            "medianBurstIRE": -9,
            "dropOuts": {
                "fieldLine": [],
                "startx": [],
                "endx": []
            },
            "decodeFaults": 0
        },
        {
            "isFirstField": false,
            "syncConf": 100,
            "seqNo": 2,
            "audioSamples": 0,
            "diskLoc": 0.0,
            "medianBurstIRE": -9,
            "dropOuts": {
                "fieldLine": [],
                "startx": [],
                "endx": []
            },
            "decodeFaults": 0,
            "vitsMetrics": {
                "whiteSNR": 96,
                "blackLineF1PreTBCIRE": 0.75,
                "blackLineF2PreTBCIRE": 0.75,
                "blackLineF1PostTBCIRE": 0.75,
                "blackLineF2PostTBCIRE": 0.75,
                "blackLineF1PSNR": 96,
                "blackLineF2PSNR": 96
            },
            "frameNumber": 52220
        }
    ]
}

[Zcooger]

Ok, let's have a look what I achieved with adjusting the levels according to ITU-R Rec.601 standard.

file.c:

static int _rf_file_write_uint16_real(void *private, int16_t *iq_data, size_t samples)
{
    rf_file_t *rf = private;
    uint16_t *u16 = rf->data;
    int i;

    while (samples)
    {
        for (i = 0; i < rf->samples && i < samples; i++, iq_data += 2)
        {
            u16[i] = (int)(iq_data[0] + 9830) * (1 + 2.0 / 3.0);
        }

        fwrite(rf->data, rf->data_size, i, rf->f);

        samples -= i;
    }

    return(HACKTV_OK);
}

Additionaly the gamma value should be flat 1.0 - the linear gradient is finaly linear on oscilloscope view now.

video.c:

const vid_config_t vid_config_pal = {

    /* Composite PAL */
    .output_type    = HACKTV_INT16_REAL,

    .level          = 1.0, /* Overall signal level */
    .video_level    = 1.0, /* Power level of video */

    .frame_rate_num = 25,
    .frame_rate_den = 1,
    .lines          = 625,
    .active_lines   = 576,
    .active_width   = 0.00005195, /* 51.95µs */
    .active_left    = 0.00001040, /* |-->| 10.40µs */

    .hsync_width       = 0.00000470, /* 4.70 ±0.20µs */
    .vsync_short_width = 0.00000235, /* 2.35 ±0.10µs */
    .vsync_long_width  = 0.00002730, /* 2.73 ±0.20µs */

    .white_level    =  0.70,
    .black_level    =  0.00,
    .blanking_level =  0.00,
    .sync_level     = -0.30,

    .colour_mode    = VID_PAL,
    .burst_width    = 0.00000225, /* 2.25 ±0.23µs */
    .burst_left     = 0.00000560, /* |-->| 5.6 ±0.1µs */
    .burst_level    = 3.0 / 7.0, /* 3 / 7 of white - blanking level */
    .colour_carrier = 4433618.75,
    .colour_lookup_lines = 625 * 4, /* The carrier repeats after 4 frames */

    .gamma          = 1.0, /* 2.8 in spec? too bright */

    .rw_co          = 0.299, /* R weight */
    .gw_co          = 0.587, /* G weight */
    .bw_co          = 0.114, /* B weight */
    .iu_co          = 0.000,
    .iv_co          = 0.877,
    .qu_co          = 0.493,
    .qv_co          = 0.000,
};

This JSON file is adjusted to HackTV output - it means all brightness, contrast, gamma and saturation levels are 99,9% accurate.

Remove ".txt" ldsample.tbc.json.txt

75% color bars (EBU 100/0/75/0): Linear slope: Grayscale: 75% color bars on oscilloscope: Final HackTV output (fields merged): Videocrypt in color - bring back firefly:

[neo7530]

Gerät work 😊 Do you think we can write a Realtime color-Decoder with stdin and stdout for using it with hacktv and vlc? The source for ld-colour-pal and ntsc is available on github. Or an vdub Filter... My c isn't strong enough... 😂

[Zcooger]

I'm realistic about this and it may take too much time, because I'm introducing myself to programming and I can't write more advanced programs yet. VirtualDub filter would be nice.

[neo7530]

Same as me. The First Thing i ever wrote in c was an ecm-generator for tsduck. It took 2 weeks for nearly 1000 lines of Code...

[neo7530]

For ld-decode you need 626 lines of raw Image. Else the decoded picture will move to the top while decoding... I wrote a small Program, that adds a blank line after 2 halfframes and writes out the json file... More to come tomorrow.

[Zcooger]

If you can handle that - another thing is worth fixing: https://github.com/fsphil/hacktv/issues/25

[neo7530]

I can implement it, no Problem. Videocrypt adds one line delay, nagravision 28 lines of first halfframe, then 2nd halfframe, so the field order is wrong.

[Zcooger]

If you look closely the WSS signal on scrambled picture should be at the odd field at the bottom because of encoder buffer so 344 lines should be skipped around the grey rectangle.

[fsphil]

What are you actually trying to do? I'm not sure I understand.

[Zcooger]

Correct the scrambled output for baseband color decoder "LD Analyse" which does not work like TV that is looking for vertical pulse (it just reads uint16 raw data as is):

[neo7530]

bin2tbc.tar.gz.txt here it is, rename to bin2tbc.tar.gz and extract… readme included. Windows & Linux build.

have fun.

[neo7530]

bin2tbc.tar.gz.txt new Linux Version with stdin (Var1 = '-' ) but if you use stdin, you have to edit the json file and close it manually, but it works...

[neo7530]

and a "bug" in ld-decomb-pal:

this change in filterthread.cpp:

// Set the first and last active scan line
firstActiveScanLine = 44;
lastActiveScanLine = 620;

will give a output frame with full 576 pix heigth...

you can pipe the *.tbc via a fifo-file directly to ffmpeg:

ld-comb-pal output.tbc rgb.fifo

ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb48 -s 728x576 -i rgb.fifo -vcodec h264 -vf scale=720:576 -f mpegts out.ts

[neo7530]

@fsphil sorry for hijacking this issue…

[Zcooger]

I close this issue, sync level is fixed by using unsigned data types (uint8, uint16). Gamma is a thing for another ticket.