ffmpeg consumer color space

[TomKaltz]

Recorded a file and played back on same channel and seems that either encode or decode color space isn't being translated correctly.

[TomKaltz]

This was tested with the h264, prores and dnxhd codecs. All with same behavior.

[TomKaltz]

Also seems to be an encoding issue since VLC exhibits the same wrong colors as when played back through CasparCG.

[TomKaltz]

@ronag Here is how MediaInfo reads the color metadata of the recorded file with default x264 encoder params.

[premultiply]

This looks like RGB mode.

[ronag]

it's YUV444P

[ronag]

@TomKaltz that's not the info we need... we need the color range/matrix etc... there are so many "color" thing it's hard to keep track of

[TomKaltz]

Seems like MediaInfo can't display that sort of information I'll see if I can figure out another way to detect it.

[premultiply]

Ok, then it should be a better solution to pass RGBA to the ffmpeg chain and let user decide what we wants to convert to. Passing any YUV format gets us in trouble to choose the correct color before it is converted to this format (and will be converted again by user to 4:2:2/4:2:0). Only RGB is indepenant of all these color things.

[premultiply]

Or is the current behavior the result of ffmpeg autoconversion from RGBA to the highest possible color resolution supported by the selected codec? In this case only the correct videofilter needs to be added by user to fix this.

[TomKaltz]

What does the mixer pass to ffmpeg consumer currently?

[premultiply]

https://github.com/CasparCG/server/blob/b95525ac909359f1ce2b22fd045e62b11e4d390f/src/modules/ffmpeg/consumer/ffmpeg_consumer.cpp#L181

AV_PIX_FMT_BGRA

[ronag]

If you output to a codec with RGB support, does it look correct?

[premultiply]

Trying to reproduce. What is the correct syntax for AMCP with videofilters?

[ronag]

@5opr4ni syntax? I don't remember off the top of my head.

[5opr4ni]

PLAY 1-12 DECKLINK DEVICE 1 FORMAT 1080i5000 vf "split[a][b];[a]smartblur=lr=5.0, colorbalance=bs=.3[blur];[b][blur] blend=all_mode=lighten" af pan=stereo|c0=c1|c1=c1 PUSH or a simpler one PLAY 1-12 DECKLINK DEVICE 1 FORMAT 1080i5000 vf "colorbalance=bs=.3" af pan=stereo|c0=c0|c1=c0 PUSH 20

[premultiply]

No this is ffmpeg producer. I try to record a file.

[5opr4ni]

Oh! Sorry! Hold on

[5opr4ni]

ADD 1 FILE dnx.mxf -codec:v dnxhd -b:v 120M -flags:v +ilme+ildct -threads:v 4 -filter:v format=yuv422p,interlace

[5opr4ni]

The vf and af are written a bit different here.

[premultiply]

yes. -filter:v format=yuv422p,interlace is the important step here to convert from RGBA input to proper 4:2:2 output. If not inserted ffmpeg will sent the best format the output codec/container can handle. In case of x264 this is yuv444p.

[premultiply]

Maybe it would be a good idea to include something like preset files that can be called with their filename for the often requiered settings/formats.

RECORDER 1-700 XDCAMHD422 test.mxf or something like this

[ronag]

If not inserted ffmpeg will sent the best format the output codec/container can handle. In case of x264 this is yuv444p.

But using yuv444p is not wrong and should not be a problem?

[ronag]

I think presets and the like is a client thing.

[premultiply]

yuv444p is autoselected because x264 does not support BGRA. BGRA is used by default. No nothing is wrong with that but it may be a little bit suprising for the "normal" user that everything need to be explicitly converted to a common format.

It's the same with the audio.

[5opr4ni]

Or in AMCP 3.0 ;)

[TomKaltz]

Yeah but has anyone confirmed correct color even with proper filter params? I won’t be able to test until later.

On Wed, Feb 28, 2018 at 08:06 Olle Soprani notifications@github.com wrote:

Or in AMCP 3.0 ;)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CasparCG/server/issues/884#issuecomment-369233238, or mute the thread https://github.com/notifications/unsubscribe-auth/AC0_IcAfW2PvKoKzyNYnR2Wtzfmevb8dks5tZU9BgaJpZM4ST7SM .

-- Thomas Kaltz III 586-214-7150 Sent from Gmail Mobile

[premultiply]

Yes. It's fine if you specify ALL filters and metadata correctly...

[ronag]

Well, we could auto add filters that are required. Just need to figure out what those are.

[premultiply]

Depends on what you like to do. Because of this I recommended something like the preset files.

There will be people who just want a 4:2:0 h264 aac stereo mp4. Other would like to have for example a MXF AVCIntra100 file with 8 audiotracks. Others would like RGBA files without any audio. One would like interlaced, the other progressive etc.

[ronag]

this has nothing to do with presets, the meta we send from caspar into ffmpeg should be correct without having to add override filters, e.g. "ADD 1 FILE asd.mov -codec:v libx264" should output correct content. It seems ffmpeg for some reason assumes that the RGBA frames are not full range (or something). That's something we should fix.

[ronag]

Maybe something like: -filter:v colorspace=itrc=srgb:irange=pc:trc=srgb:range=pc is required. @TomKaltz could you try?

[premultiply]

That is only a little part of the problem and will introduce other issues. RGBA is a good common format and if you just define a target pixel_format you like to convert to then it will be converted to that but there are much more properties you have to take care of like colormatrix which on its own depends on input AND output resolution of filter chain if converting to yuv etc.

[ronag]

That is only a little part of the problem and will introduce other issues.

I don't follow. What problems does it not resolve and what other issues does it introduce?

[premultiply]

Example command (Audio don't care) ADD 1-700 FILE "test.mp4" -codec:v libx264 -flags:v ilme+ildct -filter:v interlace,scale=out_range=tv:out_color_matrix=bt709,format=pix_fmts=yuv422p -preset:v veryfast -b:v 25M

Order of filters are important for and parameters are dependent on desired output.

[ronag]

And auto prepending colorspace=itrc=srgb:irange=pc:trc=srgb:range=pc to that is a problem because?

Not trying to be difficult :). I just don't get it.

[ronag]

Ah, I think I get it. You are suggesting we auto set output color settings as well when we know that the output is e.g. HD and YUV?

[premultiply]

Yes. Tagging the RGB input to ffmpeg does not change anything. ffmpeg already knows that this is RGBA fullrange.

The problem is the user defined destination format which needs different properties on different output sizes, framerates, containers etc. This does not necessarily depend on channel resolution or is always a yuv format. And because this can be very complicated to do it right I suggest to maybe drop the whole user defined filtering here from AMCP command and move it to preset files on server instead which can be called via AMCP as something like RECORD 1-1 presetname filenname.ext. Some of the most common file format presets should be build/packaged/delivered with server. This would keep the complexity low for users and developers and it is very easy to extend or modify it for people who want to dive into.

[ronag]

Ah, so the OP's "bug" is actually a question of parameters. @TomKaltz can you try with something like:

-filter:v colorspace=all=bt709

@premultiply: we can continue this on slack

[ronag]

@premultiply pointed out colorspace doesn't to rgb, so you should do:

-filter:v scale=out_matrix=bt709:out_range=bt709,format=yuv422p

[premultiply]

no it's -filter:v scale=out_color_matrix=bt709:out_range=tv,format=yuv422p

[TomKaltz]

We need to add a ffmpeg consumer recipe wiki page or like @premultiply suggested perhaps add a preset feature.

[ronag]

well, unless someone wants to develop and maintain presets... I think wiki is the way to go for now...

[premultiply]

I could create and maintain some basic presets included in the server package. That would not be a problem. These would be just simple text files with parts of the ffmpeg command lines.

[TomKaltz]

Someone would have to implement the preset system. PR welcome. For now we should add this to...

https://github.com/CasparCG/server/wiki/FFMPEG-Consumer

[TKooijmans]

I like the idea of adding working presets already to the server. I would be great if a XML file with predefined presets is present in de CasperCG folder like the config file. If a user wants to change or add new presets it can be easy done there! If all the ffmpeg options and settings are in there in the right order the the user can try other settings without reading all the FFmpeg stuff.

[premultiply]

No need to have complicated XML files. Just simple plain text.

[TKooijmans]

If it's a easy readable file it's fine but with xml, you split Presetname, video and audio settings.. If put you a preset on every line it's fine too.