Color metadata is not being retained at each step of the decoding process

[mitcoding]

In theory, tools like ld-chroma-decode should be able to embed the color properties of the decode so that tools like ffmpeg know if it's NTSC or PAL SD colorspace. You can see if this metadata exists by running this ffprobe command:

ffprobe -v error -show_streams input_file.ext | grep -i color

expected output if: PAL SD: color_range=tv color_space=bt470bg color_transfer=bt470bg color_primaries=bt470bg

NTSC SD: color_range=tv color_space=smpte170m color_transfer=smpte170m color_primaries=smpte170m

If we aren't able to embed this data in each of the generated files from the different encode steps then we need to make sure our ffmpeg and/or VaporSynth code examples define these on the input side of the conversion command. Either way will still need to update our Encoding examples to set the Color metadata properties for files generated by ffmpeg or VaporSynth on the output side.

PAL (this example assumes ld-chroma-decoder can add the metadata to the piped file to ffmpeg. if we can't do that then the new line 5 also needs to be added between lines 2 and 3):

@@ 4,5 +5,6 @@
    ld-chroma-decoder input.tbc | ffmpeg -f s16le -ar 44.1k -ac 2\
    -i input.pcm -f rawvideo -r 25 -pix_fmt rgb48 -s 928x576\ 
    -i - -pix_fmt yuv420p -vcodec libx264 -crf 18 -flags +ildct+ilme\ 
    -aspect 768:576\
+  -color_range 1 -color_primaries bt470bg -color_trc gamma28 -colorspace bt470bg
    output.576i25.mp4

NTSC (this example assumes ld-chroma-decoder can add the metadata to the piped file to ffmpeg. if we can't do that then new line 5 also needs to be added between lines 1 and 2):

@@ 4,5 +5,6 @@
    ld-chroma-decoder -qf ntsc2d input.doc.tbc | ffmpeg -f rawvideo\
    -r 30000/1001 -pix_fmt rgb48 -s 760x488 -i - -f s16le -r 44.1k -ac 2\
    -i input.efm.pcm -vcodec ffv1 -acodec flac -compression_level 11\ 
    -pix_fmt yuv444p -aspect 4:3\
+   -color_range 1 -color_primaries smpte170m -color_trc smpte170m -colorspace smpte170m\
    output.mkv

[atsampson]

The output from ld-chroma-decoder is raw 16-bit full-range RGB (-f rawvideo -pix_fmt rgb48 -color_range jpeg in ffmpeg terms), so there's no opportunity to embed metadata in it; adding these options to ffmpeg here as you suggest should work.

In practice, the difference between the primary coordinates in the two standards, which is on the order of 1%, is probably too small to make a difference in practice at the moment - the HF rolloff we're seeing on LaserDiscs varies more than that between discs. But it'd be nice to get this right for when the tools are fed with higher-quality source material.

(It might also be worth noting that Poynton p269 is skeptical about whether PAL video equipment has ever actually used gamma 2.8: "In fact, European displays are comparable to displays in other parts of the world, and encoding to Rec. 709 is appropriate.")

[mitcoding]

@atsampson, that is interesting info about gamma 2.8, Rec. 709 and PAL displays . When you say we output 16-Bit Full Range RGB. Do you mean the tools encode black at 0 and white at 255, instead of 16 and 235? If so that is still important information as we want our examples to set -color_range to 2 for the input file.

The main reason I raised this issue is to make sure the correct metadata is set so consumer displays or further digital manipulation in other tools can render the video as accurately as possible. Thus, if we aren't actually decoding them as smpte1070 or bt470bg then we should set the flags to whatever colorspace we are decoding them as and make sure that we let our users know what colorspace our tools output so if they do any further processing they know what to expect from the input file.

[atsampson]

The RGB output is 16-bit, so the range is 0-65535, where 0 and 65535 correspond to the black and white levels in the JSON metadata. See RGB::convertLine (NTSC) and PalColour::decodeLine (PAL) in ld-chroma-decoder.

[mitcoding]

After thinking about this further, I think I might have been thinking about this problem incorrectly for the first decoding step. So I'm throwing out this new thought, to make sure that changing my mind is actually getting us closer to the truth of the mater and not further.

I believe my faulty assumption was that Laserdiscs encoded their signal using either Rec 601 for NTSC or bt470bg for PAL. But I believe that is a miss step as those standards where created specifically for conversion from Analogue to Digital, and were a compromise as the digital systems of that day couldn't reproduce all the colors in the pure analogue systems. In which case the data stored on the RF signal we capture should have the potential to display all the color that the analogue NSTC or PAL systems had which is a wider color gamut then even Rec 709. So I wonder if it would be safer to make sure we over sample and encode the raw RGB output with Rec 2020 color gamut, even though the Analogue NTSC or PAL standards don't include all the colors available in that spec, however they do contain a full sub set of the colors available in Rec 2020. This change in idea is assuming the chart I'm reading here is correct:

Thus assuming the above chart is correct it could be beneficial to encode the raw rgb file generated by ld-chorma-decode, to output full RGB with the Rec.2020 color gamut. Which means, the colorspace flags for the input context would use the Rec.2020 flags (there are couple different ones to choose from as they deal with different transfer functions so we would want to pick the one that matches laserdiscs gamut encoding best).

Based on what I saw in the ld-chroma-decode code, I would assume we would have to change the calculation for full red, green, and blue so that the uv coordinates for those colors matched the position that analogue NTSC/PAL full red, green, and blue fall on the rec 2020 color gamut set. Is this a correct assumption? or even doable?

[mitcoding]

As I think about this further after writing this, I think I've still got wrong thinking especially with use of Rec 2020 but only mapping the NTSC/PAL full red green blue points to where they would actually fall with in the rec 2020. Because if a user needs to down convert to rec 709 or rec 601 for their display needs it would cause the red, green and blue to shift to far down using the current tools like ffmpeg or VapourSynth as they would assume our red, green, blue points aren't full red, green blue and compensate accordingly.

My assumption after thinking about this further especially since our pal decoder is based on Jim Easterbrook's pal decoder, that our code is writing to the SD PAL colorspace bt.470bg since Jim Easterbook's pal decoder was for authoring the Doctor Who DVDs which are bt.470bg.

To summarize all i was looking for is what colorspace are we decoding to when ld-chomra-decode generates the raw rgb file, so I can set the correct colorspace flags for both the input and output side.

If there are any other ideas in my hair brained thought experiment that can help improve the color preservation we can open another issue as an enhancement. I would also appreciate it if someone took the time to either explain where my thought experiment went wrong or right. Either by typing me an explanation or pointing me to the materials I should read so I can figure out where I went wrong.

[atsampson]

@mitcoding, I would very strongly recommend that you read Charles Poynton's "Digital Video and HDTV: Algorithms and Interfaces" - the first edition, which has more material on analogue video and how it corresponds to digital standards, and the best coverage of colour representation that I've seen in any book about video, is readily available secondhand. (You'll see various references to this book in the bits of the ld-chroma-decoder code that deal with colourspace conversion.)

[mitcoding]

@atsamspon Thanks for the tip on the book. I've got the book on order from thriftbooks.

With that said, so we can close this issue as completed. Can anyone clarify exactly which colorspace ld-chroma-decode is using to encode the raw RGB48 video file to let tools like ffmpeg, VapourSynth and Kodi know how to decode the video using the correct colorspace, transfer function, and color matrix?

[simoninns]

Closing this issue as this isn't really an issue. @mitcoding - atsampson's suggestion of studying Poynton is based on the fact that your question (in the context of the chroma decoder) doesn't really make sense - so it's not possible to answer. The output is RGB48 - ffmpeg or any other tool doesn't need to know anything further as there is no decoding to perform.