PAL TBC output is not 4xfsc

[simoninns]

While investigating the possibility of using the PAL transform decoder it became apparent that the output from ld-decode is not exactly 4xfsc (and this generates issues with video processing that expects output to be exactly on spec):

1135 x 625 x 25 = 17.734375 4.43361875 x 4 MHz sampling = 17.734475 MHz

The JSON currently indicates that the sampling is the latter; but the output seems to be the former. This will need correction in order to allow clean integration with external PAL tools.

[happycube]

One thought I've just had is to output 1135px of 4xfsc and drop the last .0064 or so pixels, nobody will miss a tiny bit of the back porch, right? ;)

[rtrussell]

(copied from Facebook Messenger as requested) I would suggest you store the 4 extra samples in the '626th' line of the frame. AIUI this is currently a duplicate of the first line in the next frame, but presumably it need not be. The odd and even fields will end up being displaced from each other by two samples; this is 'correct' (and what the Transform PAL Decoder expects) but might surprise some people!

[simoninns]

Additional, useful information on the PAL format is available from this document: https://tech.ebu.ch/docs/tech/tech3280.pdf

[skunge2000]

Yes - EBU 3280 which defines the 4fsc 'D3' and 'D2' PAL formats I believe states :

As a consequence of this non–orthogonal structure, two extra samples are needed per field. These are located on lines 313 and 625 and are numbered 1135 and 1136; they appear immediately prior to the first active picture sample, 0000. These extra samples do not affect the continuous signal concept where all but two lines in a field have 1135 samples and the other two have 1136.

As the .tbc format is already following EBU 3280 in level terms (black/blanking at 8-bit 64 / 16-bit 16384 and reference white at 8-bit211/16-bit 54016) it may make sense to follow EBU 3280 in this respect too?

[rtrussell]

As the .tbc format is already following EBU 3280 in level terms ... it may make sense to follow EBU 3280 in this respect too?

That would imply abandoning the orthogonality of the TBC file format (instead of all lines being 1135 samples long, one line at the end of the first field would need to be 1137 samples long). From my perspective that's too high a price to pay for strict Tech.3280 compliance; I'd rather keep all the lines the same length, as now, and consider the four extra samples to be at the end of the frame where they can be moved into the 626th line (currently a duplicate of the first line in the next frame).

[skunge2000]

As the .tbc format is already following EBU 3280 in level terms ... it may make sense to follow EBU 3280 in this respect too?

That would imply abandoning the orthogonality of the TBC file format (instead of all lines being 1135 samples long, one line at the end of the first field would need to be 1137 samples long). From my perspective that's too high a price to pay for strict Tech.3280 compliance; I'd rather keep all the lines the same length, as now, and consider the four extra samples to be at the end of the frame where they can be moved into the 626th line (currently a duplicate of the first line in the next frame).

Yes - as long as we know where the 4 samples are - and not having them spread between field 0 and field 1 (or field 1 and field 2 as I'm sure we used to call them) doesn't cause things to get 'out of sync' then yes - that makes sense.

Thinking about it do the samples actually need to be stored at all? We need the signal to be precisely 4fsc sampled - but do we actually need to retain those 2 extra samples per field for further processing (i.e. they need to be sampled to get us to precisely 4fsc, but do they need to be stored?)

(Or am I missing something?)

[rtrussell]

as long as we know where the 4 samples are - and not having them spread between field 0 and field 1 (or field 1 and field 2 as I'm sure we used to call them) doesn't cause things to get 'out of sync'

The division of the frame into 1135-sample 'lines' is purely for convenience, it's a continuous data stream so as long as all the samples are present, in the right order, nothing should get 'out of sync'.

do the samples actually need to be stored at all?

I would say yes, because they are part of the video stream decoded from the disc, so the .TBC file is not doing its job as a container for that data if any samples are irretrievably discarded: each frame would last only 39.99977... ms rather than 40!

[skunge2000]

as long as we know where the 4 samples are - and not having them spread between field 0 and field 1 (or field 1 and field 2 as I'm sure we used to call them) doesn't cause things to get 'out of sync'

The division of the frame into 1135-sample 'lines' is purely for convenience, it's a continuous data stream so as long as all the samples are present, in the right order, nothing should get 'out of sync'.

do the samples actually need to be stored at all?

I would say yes, because they are part of the video stream decoded from the disc, so the .TBC file is not doing its job as a container for that data if any samples are irretrievably discarded: each frame would last only 39.99977... ms rather than 40!

Yes - that makes total sense, and avoids having to insert 4 extra samples per frame back in when using the file as a source downstream.

[simoninns]

Owing to lack of material to test and develop this with; pushing to rev7 as agreed on IRC

[happycube]

I'll probably implement this as a command line option soon. One interesting note is that if output is out of sync with the colour burst, it's possible for yellow to cross into the illegal area in the standard digital levels...

[skunge2000]

Yes - that's why the PAL 4fsc standard samples at the burst phase it does. PAL 4fsc samples at 45, 135, 225 and 315 phases of subcarrier to avoid sampling at the peak which with 100% yellow is out of range of the quantisation standard... (If you sample at 0, 90, 180, 270 you go out of range I think) As per https://tech.ebu.ch/docs/tech/tech3280.pdf and Poynton.

Are we agreed that the 4 additional samples that will be needed each frame for 4fsc locked PAL sampling should go into what is effectively line 626 within the .tbc file structure?

This will keep the sample structure 'frame repeating' as per the broadcast 4fsc spec, keeps 625 lines all 1135 samples long, and stores the additional 4 samples per frame in line 626 (first 4 samples of that line - with the rest padded with an illegal level such as 00.0 or FF.C? Or something else?).

[atsampson]

EBU Tech 3280 also has a very clear description of how the sample numbering should correspond to the analogue and digital active area, and this is somewhat different from the timings ld-decode currently uses. If ld-decode switches to producing 4fSC output for PAL, it'd be good to fix this at the same time.

[atsampson]

Copying a comment here from email before I forget about it. Here's how ld-chroma-encoder now does 4fSC output:

For subcarrier-locked output in the .tbc file, I've tried to stick to Tech 3280 as closely as possible. Each line contains the 187 blanking samples followed by the 948 active samples (with 922 samples centred in that for the 4:3 image). As before, the "626th" line contains the 4 leftover samples and 1131 samples of padding. This means that 0H (midpoint of the horizontal sync point) occurs 9.5 samples into line 1. In the .tbc.json metadata, the colourburst and active positions are given for line 1.

This keeps the sample numbering simple, and it doesn't seem to present any major difficulties, so I'm going to suggest this layout for 4fSC output from ld-decode unless anyone's got any major objections. (?)