Extract Review: Support images/videos with only named channels

BigRoy commented 2 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior:

Currently when a review is trying to get generated for a publish using e.g. a multilayer EXR with only named channels the ExtractReview logic fails:

For example, take this log: error.log From discussion here

Which shows it failing (in Deadline) on an EXR sequence that has named channels:

AO
CryptoAsset
CryptoMaterial
CryptoObject
Depth
DiffCol
DiffDir
Emit
Normal
UV
rgba

One could argue the rgba sensibly is the one to use here - maybe there's a way we can force ffmpeg to allow that as the main layer? If no dedicated R G B A channels are found.

I also wonder what makes it so that a lot of image viewers are capable of defaulting to viewing the RGBA by default instead of the other channels - if there's some sort of metadata or logic involved with knowing which of the named channels is the default viewable then we should adhere to that to in the hope FFMPEG can process it too.

Expected Behavior:

Allow these sequences to:

Not error in the publishing process
Generate correct reviewables.

Version

1.0.0

What platform you are running on?

Windows

Steps To Reproduce:

This can be reproduced from Blender according to this:

Set up blender render scene with multilayer outputs

ayon_blender_render

and from: ayon+settings://blender/RenderSettings/renderer?project=test

ayon_blender_render_settings

Are there any labels you wish to add?

[X] I have added the relevant labels to the bug report.

Relevant log output:

No response

Additional context:

See relevant Ayon discord thread

BigRoy commented 2 months ago

According to this Blender Artists topic we should be able to pass a -layer argument to FFMPEG to have it force pick a named channel of our choice.

This may then be e.g. ffmpeg -layer rgba or whatever but I can't find much info about that -layer argument.

BigRoy commented 2 months ago

Example EXR that fails here:

ch_carole_001_renderShadingMain.beauty.1001.zip

BigRoy commented 2 months ago

For what it's worth, the -layer argument does work and needs to be passed before the -i (input) argument.

So this would work:

ffmpeg -layer rgba -i ch_carole_001_renderShadingMain.beauty.1001.exr test.mp4

Example log then would be:

ffmpeg version 6.1.1-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[exr @ 000001cd9f021780] Channel doesn't match layer : AO.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : AO.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : AO.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : AO.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset00.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset00.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset00.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset00.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset01.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset01.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset01.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset01.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset02.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset02.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset02.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoAsset02.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial00.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial00.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial00.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial00.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial01.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial01.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial01.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial01.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial02.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial02.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial02.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoMaterial02.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject00.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject00.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject00.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject00.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject01.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject01.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject01.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject01.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject02.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject02.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject02.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : CryptoObject02.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : Depth.V.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffCol.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffCol.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffCol.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffCol.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffDir.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffDir.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffDir.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : DiffDir.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : Emit.A.
[exr @ 000001cd9f021780] Channel doesn't match layer : Emit.B.
[exr @ 000001cd9f021780] Channel doesn't match layer : Emit.G.
[exr @ 000001cd9f021780] Channel doesn't match layer : Emit.R.
[exr @ 000001cd9f021780] Channel doesn't match layer : Normal.X.
[exr @ 000001cd9f021780] Channel doesn't match layer : Normal.Y.
[exr @ 000001cd9f021780] Channel doesn't match layer : Normal.Z.
[exr @ 000001cd9f021780] Channel doesn't match layer : UV.X.
[exr @ 000001cd9f021780] Channel doesn't match layer : UV.Y.
[exr @ 000001cd9f021780] Channel doesn't match layer : UV.Z.
[exr @ 000001cd9f021780] Channel match layer : rgba.A.
[exr @ 000001cd9f021780] Channel match layer : rgba.B.
[exr @ 000001cd9f021780] Channel match layer : rgba.G.
[exr @ 000001cd9f021780] Channel match layer : rgba.R.
Input #0, exr_pipe, from 'ch_carole_001_renderShadingMain.beauty.1001.exr':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: exr, gbrapf32le(unknown/unknown/linear), 432x324 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn
File 'test.mp4' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (exr (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[exr @ 000001cd9f014880] Channel doesn't match layer : AO.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : AO.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : AO.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : AO.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset00.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset00.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset00.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset00.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset01.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset01.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset01.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset01.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset02.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset02.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset02.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoAsset02.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial00.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial00.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial00.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial00.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial01.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial01.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial01.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial01.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial02.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial02.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial02.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoMaterial02.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject00.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject00.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject00.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject00.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject01.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject01.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject01.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject01.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject02.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject02.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject02.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : CryptoObject02.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : Depth.V.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffCol.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffCol.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffCol.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffCol.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffDir.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffDir.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffDir.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : DiffDir.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : Emit.A.
[exr @ 000001cd9f014880] Channel doesn't match layer : Emit.B.
[exr @ 000001cd9f014880] Channel doesn't match layer : Emit.G.
[exr @ 000001cd9f014880] Channel doesn't match layer : Emit.R.
[exr @ 000001cd9f014880] Channel doesn't match layer : Normal.X.
[exr @ 000001cd9f014880] Channel doesn't match layer : Normal.Y.
[exr @ 000001cd9f014880] Channel doesn't match layer : Normal.Z.
[exr @ 000001cd9f014880] Channel doesn't match layer : UV.X.
[exr @ 000001cd9f014880] Channel doesn't match layer : UV.Y.
[exr @ 000001cd9f014880] Channel doesn't match layer : UV.Z.
[exr @ 000001cd9f014880] Channel match layer : rgba.A.
[exr @ 000001cd9f014880] Channel match layer : rgba.B.
[exr @ 000001cd9f014880] Channel match layer : rgba.G.
[exr @ 000001cd9f014880] Channel match layer : rgba.R.
[libx264 @ 000001cd9f066000] using SAR=1/1
[libx264 @ 000001cd9f066000] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 000001cd9f066000] profile High 4:4:4 Predictive, level 2.1, 4:4:4, 10-bit
[libx264 @ 000001cd9f066000] 264 - core 164 r3172 c1c9931 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=10 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=81 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf60.16.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv444p10le(tv, unknown/unknown/linear, progressive), 432x324 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 12800 tbn
    Metadata:
      encoder         : Lavc60.31.102 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[out#0/mp4 @ 000001cd9f010ec0] video:3kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 29.601120%
frame=    1 fps=0.0 q=40.0 Lsize=       4kB time=00:00:00.00 bitrate=N/A speed=   0x
[libx264 @ 000001cd9f066000] frame I:1     Avg QP:34.75  size:  2169
[libx264 @ 000001cd9f066000] mb I  I16..4:  3.0% 82.4% 14.6%
[libx264 @ 000001cd9f066000] 8x8 transform intra:82.4%
[libx264 @ 000001cd9f066000] coded y,u,v intra: 17.9% 14.8% 6.3%
[libx264 @ 000001cd9f066000] i16 v,h,dc,p: 12% 41% 18% 29%
[libx264 @ 000001cd9f066000] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 15% 48%  4%  3%  2%  3%  1%  2%
[libx264 @ 000001cd9f066000] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 24% 21%  7%  8%  3%  6%  2%  6%
[libx264 @ 000001cd9f066000] kb/s:433.80

So, as mentioned here:

We could use the extracted metadata of the available channels to write some extra logic that if there are no main channels to allow fallback in our codebase by default.

So that e.g. we can have an ordered list as fallback like: rgba, depth or whatever. And furthermore we could even allow it to fallback on ANY named channel to avoid an error whatsoever, but we could log a warning that we're then just picking any named channel and that one may need to configure a certain setting to set a "preferred" channel?

ynput / ayon-core