vulkan_sdl2 colors inverted

[alatteri]

see the attached photo. on the left is the original signal, on the right is UG with vulkan_sdl2. standard SDL is fine though.

[2022-12-06 21:15:47] UltraGrid 1.7+ (tags/continuous rev f213c06 built Dec  6 2022 20:02:46)
[2022-12-06 21:15:47] 
[2022-12-06 21:15:47] Display device   : vulkan_sdl2
[2022-12-06 21:15:47] Capture device   : none
[2022-12-06 21:15:47] Audio capture    : none
[2022-12-06 21:15:47] Audio playback   : none
[2022-12-06 21:15:47] MTU              : 9000 B
[2022-12-06 21:15:47] Video compression: none
[2022-12-06 21:15:47] Audio codec      : PCM
[2022-12-06 21:15:47] Network protocol : UltraGrid RTP
[2022-12-06 21:15:47] Audio FEC        : none
[2022-12-06 21:15:47] Video FEC        : none
[2022-12-06 21:15:47] 
[2022-12-06 21:15:47] [SDL] Using driver: x11
[2022-12-06 21:15:47] SDL2 initialized successfully.
[2022-12-06 21:15:47] [VULKAN_SDL2] Path to shaders: "/usr/local/bin/../share/ultragrid/vulkan_shaders"
[2022-12-06 21:15:47] MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0
[2022-12-06 21:15:47] 
[2022-12-06 21:15:47] [VULKAN_SDL2] yCbCr feature supported.
[2022-12-06 21:15:47] [VULKAN_SDL2] Vulkan uses GPU called: NVIDIA GeForce GTX 1650
[2022-12-06 21:15:47] [VULKAN_SDL2] Used Vulkan API: 1.1
[2022-12-06 21:15:47] [VULKAN_SDL2] Recreating swapchain, size: 3840x2160, format: A2B10G10R10UnormPack32
[2022-12-06 21:15:47] [VULKAN_SDL2] Vulkan display initialised.
[2022-12-06 21:15:47] [VULKAN_SDL2] Recreating render_pipeline
[2022-12-06 21:15:47] Control socket listening on port 41439
[2022-12-06 21:15:47] [key control] Stdin is not a TTY - disabling keyboard control.
[2022-12-06 21:16:11] [VULKAN_SDL2] Supported codecs are: RGBA UYVY YUYV Y216 Y416 R10k 
[2022-12-06 21:16:11] [video dec.] New incoming video format detected: 2048x1080 @24.00p, codec H.265
[2022-12-06 21:16:11] [lavd] Using decoder: hevc
[2022-12-06 21:16:12] [video dec.] Detected internal codec: R12L
[2022-12-06 21:16:12] [video dec.] Successfully reconfigured display to 2048x1080 @24.00p, codec R10k
[2022-12-06 21:16:14] [lavd] Using decoder: hevc

[MartinPulec]

Do you have some more information about that? from format: A2B10G10R10UnormPack32 I believe you use 10-bit pipeline. I've tested:

uv -t testcard:codec=R10k     # looked ok; with the conversions that you probably use:
uv -t testcard:codec=R12L -c libavcodec:encoder=libx265 -d vulkan_sdl2     # also correct

using 10-bit pipeline as well (but only 8-bit display), the output was correct - bars R,G,B,Y,C,M. I was using two setups with X11 and NV 1080 (driver 470) vs 2080 Ti (driver 525). Are the colors swapped for you with the above as well?

[alatteri]

OK... I have been adding setting the Xorg to use 30 bit color via:

Section "Screen"
    Identifier "Screen"
    DefaultDepth 30
EndSection

in /usr/share/X11/xorg.conf.d/special.conf and also by running xrandr before UG xrandr --output HDMI-2 --set "max bpc" 12

that is likely where format: A2B10G10R10UnormPack32 comes from.

removing those 2 configurations returns the color to normal. I'll try to do some more evaluations with greyscale patterns to see if there is a benefit to trying to force 10bit. But if the above gives you a better context to fix things... then great.

[MartinPulec]

that is likely where format: A2B10G10R10UnormPack32 comes from.

Sure, that's ok, I thought so. We were trying also with 30 display depth but were not able to reproduce that. We were, however not able to issue xrandr --output HDMI-0 --set "max bpc" 12 (HDMI-0 in our case). Is it somehow important? The thing is that seemingly the 10-bit output runs without that, anyways. Although the display, we were using, seems to output 8-bits, NVidia mitigates that with dithering (so the output "looks" 10 bit, although it isn't). No color inversion occurred for us (but of course, the setup is quite different).

[alatteri]

Hi Martin... it seems the issue originates via the xorg.conf.d/special.conf entries. --set "max bpc" 12 doesn't affect anything one way or another.

[alatteri]

OK....so I took the HDMI output of the receiver and put it into a BMD running ScopeBox on Mac. I then fed a 0-1024 black to white ramp into the encoder. I then had the receiver set to "no 10bit force" and then the "10 bit force" as described above. You can see that forcing 10bit does work accurately (smooth gradations) and makes a huge difference in color accuracy. So it would be great to get the color swapping problem fixed.

Thanks.

[mpiatka]

Hi, we are unfortunately still unable to reproduce the issue. I've tried on another computer (ubuntu 22.04, 1080 Ti, driver 510.108) with a native 10-bit display, but the colors are correct over here.

In order to attempt to narrow down the problem could you please try the following test cases and note if the color bars are in the correct order (should be red, green, blue, yellow, cyan, magenta from left to right).

1. uv -t testcard:1920:1080:15:RGB -d vulkan_sdl2
2. uv -t testcard:1920:1080:15:R10k -d vulkan_sdl2

Also, Am I correct in understanding that the colors are only wrong when X11 is configured to output 10-bit?

[alatteri]

Hi Martin,

I'll try those different test cards later today, but yes, the colors are only inverted when configuring X11 to be 10bit via the configuration file. The xrandr parameter doesn't make any difference. This may happening only when using lib_svthevc compression. I'll test x265 when I do the other tests.

[alatteri]

I've tried testcard, RGB/R10k/R12L, libsvt_hevc and libx265. The test card was always the same, seemingly fine. See photo below.

The swapped colors (Red and Blue) **only appear when the input is SDI**. I've tried both HD and UHD resolutions with v210,R10k and R12L inputs. Both libsvt_hevc and libx265. Color are always swapped if input is SDI. Disabling the X11 10bit force, or switching to -d sdl results in non-swapped colors, but at the lower fidelity 8bits.

Receiver: uv -d vulkan_sdl2:fs --param use-hw-accel Encoder: ./UltraGrid.AppImage --tool uv -m 1316 -t decklink:device=0:codec=R12L -c libavcodec:encoder=libsvt_hevc:qp=20 -P 5004 swapped colors:

testcard:

./UltraGrid.AppImage --tool uv -m 1316 -t decklink:device=0:codec=R12L -c libavcodec:encoder=libsvt_hevc:qp=20 -P 5004

Display device   : none
Capture device   : decklink
Video compression: libavcodec:encoder=libsvt_hevc:qp=20

[Decklink capture] Using codec: R12L
[DeckLink capture] Using device DeckLink 4K Pro
[DeckLink capture] bmdDeckLinkConfigCapturePassThroughMode set to: 1885628787
The desired display mode is supported: 1080p23.98
[DeckLink capture] Enable video input: 1080p23.98
[DeckLink] Trying to autodetect format.
Control socket listening on port 45179
Frame received (#0) - No input signal detected
[Decklink capture] Format change detected (display mode - RGB444, 12bit).
[Decklink capture] Using codec: R12L
[DeckLink capture] Enable video input: 1080p24
[lavc] Using codec: H.265, encoder: libsvt_hevc
[lavc] Setting bitrate to 7962.6 kbps.
[lavc] Warning: Unable to find suitable preset for encoder libsvt_hevc.
SVT [version]:  SVT-HEVC Encoder Lib v1.5.1
SVT [build]  :  GCC 7.5.0    64 bit
LIB Build date: Dec  9 2022 10:29:38
-------------------------------------------
[lavc libsvt_hevc @ 0x7f4d600019c0] Rext Profile forced for 422 or 444
Number of logical cores available: 32
Number of PPCS 36
Number of threads 96
------------------------------------------- 
SVT [config]: MainEXT Profile   Tier (auto) Level (auto)    
SVT [config]: EncoderMode / Tune                            : 7 / 1 
SVT [config]: EncoderBitDepth / CompressedTenBitFormat / EncoderColorFormat         : 10 / 0 / 3
SVT [config]: SourceWidth / SourceHeight / InterlacedVideo              : 1920 / 1080 / 0
SVT [config]: Fps_Numerator / Fps_Denominator / Gop Size / IntraRefreshType         : 24 / 1 / 20 / 0
SVT [config]: HierarchicalLevels / BaseLayerSwitchMode / PredStructure          : 3 / 0 / 0 
SVT [config]: BRC Mode / QP / LookaheadDistance / SceneChange               : CQP / 20 / 0 / 1 
SVT [config]: BitRateReduction / ImproveSharpness                   : 0 / 0 
SVT [config]: tileColumnCount / tileRowCount / tileSliceMode / Constraint MV        : 4 / 4 / 1 / 1
SVT [config]: De-blocking Filter / SAO Filter                       : 1 / 1 
SVT [config]: HME / UseDefaultHME                           : 1 / 1 
SVT [config]: MV Search Area Width / Height                         : 16 / 7 
SVT [config]: HRD / VBV MaxRate / BufSize / BufInit                 : 0 / 0 / 0 / 90
------------------------------------------- 

SVT [WARNING] Elevated privileges required to run with real-time policies! Check Linux Best Known Configuration in User Guide to run application in real-time without elevated privileges!

[lavc] Selected pixfmt: yuv444p10le
[lavc] Selected pixfmt has not 4:2:0 subsampling, which is usually not supported by hw. decoders
[DeckLink capture] 113 frames in 5.0161 seconds = 22.5275 FPS
[DeckLink capture] 120 frames in 5.00005 seconds = 23.9998 FPS
[DeckLink capture] 120 frames in 5.00002 seconds = 23.9999 FPS

[alatteri]

Hey guys.... Happy New Year. Any thoughts on this issue? Only happens when the input is SDI... the testcards seem fine.

[MartinPulec]

Hi @alatteri, well, yes, something.

First of all, the testcard doesn't look fine, if you look more closely, there is the order blue-green-red, but it should be red-green-blue as it should be (the same holds for the remaining "mixed" colors). So can you confirm that it renders the blue/red swapped order with simple `-t testcard:codec=RGB -d vulkan_sdl2' (and UYVY)?

Anyways, we managed somehow to reproduce the issue with combination of NVIDIA and AMD. The thing is that seemingly AMD driver doesn't render 10-bit pipeline, so when 10-bit display is connected to AMD and there is a NVIDIA present, it renders over NVIDIA and displays with AMD, concretely running uv -t testcard -d vulkan_sdl2 | grep 'Vulkan uses':

[VULKAN_SDL2] Vulkan uses GPU called: NVIDIA GeForce GTX 1080

(when we remove then NVIDIA card, Vulkan refuses to work on AMD if set to 10-bit because the pipeline doesn't support that)

So the question to move forward is what graphic card(s) do you have on system, which is the displaying card and is it the one that is rendering? Also could you attach output of (initialization of), let say, uv -t testcard:codec=R10k -d vulkan_sdl2? Thanks, M.

[alatteri]

I'll try this weekend. I know one of the test machines has a GTX1650. I will also try it on Intel NUC12 with integrated Xe graphics.

[alatteri]

Hi guys,

I did some testing today.

The first system is an off brand Chinese unit (Topton) that has a built in GTX 1650, in a relatively small form factor. Using Vulkan results in the swapped channels as per the origination of this issue, wether testcard is R10k, RGB, or UYVY. Otherwise, the luminance seems OK.

The second system is Intel NUC12. The results are different, but also problematic. With this unit, the channels are not swapped, but some values display corrupted and invalid. I also think there is a luminance brightening issue, but hard to tell with the corruption. See attached photos. I also, tried removing the 10bit Xorg conf override and the result were the same. The NUC would be my preferred platform, as it is smaller, and from a more reputable manufacturer.

Topton

uv -t testcard:codec=R10k -d vulkan_sdl2

[2023-01-07 16:43:03] UltraGrid 1.8+ (tags/continuous rev 910a522 built Jan  7 2023 16:34:48)
[2023-01-07 16:43:03] 
[2023-01-07 16:43:03] Display device   : vulkan_sdl2
[2023-01-07 16:43:03] Capture device   : testcard
[2023-01-07 16:43:03] Audio capture    : none
[2023-01-07 16:43:03] Audio playback   : none
[2023-01-07 16:43:03] MTU              : 9000 B
[2023-01-07 16:43:03] Video compression: none
[2023-01-07 16:43:03] Audio codec      : PCM
[2023-01-07 16:43:03] Network protocol : UltraGrid RTP
[2023-01-07 16:43:03] Audio FEC        : none
[2023-01-07 16:43:03] Video FEC        : none
[2023-01-07 16:43:03] 
[2023-01-07 16:43:03] [SDL] Using driver: x11
[2023-01-07 16:43:03] SDL2 initialized successfully.
[2023-01-07 16:43:03] [VULKAN_SDL2] Path to shaders: "/usr/local/bin/../share/ultragrid/vulkan_shaders"
[2023-01-07 16:43:03] [VULKAN_SDL2] yCbCr feature supported.
[2023-01-07 16:43:03] [VULKAN_SDL2] Vulkan uses GPU called: NVIDIA GeForce GTX 1650
[2023-01-07 16:43:03] [VULKAN_SDL2] Used Vulkan API: 1.1
[2023-01-07 16:43:03] [VULKAN_SDL2] Recreating swapchain, size: 960x540, format: A2B10G10R10UnormPack32
[2023-01-07 16:43:03] [VULKAN_SDL2] Vulkan display initialised.
[2023-01-07 16:43:03] [VULKAN_SDL2] Recreating render_pipeline
[2023-01-07 16:43:03] [testcard] capture set to 1920x1080 @50.00i, codec R10k, bpc 10, pattern: bars, audio off
[2023-01-07 16:43:03] Control socket listening on port 37677
[2023-01-07 16:43:03] [key control] Stdin is not a TTY - disabling keyboard control.
[2023-01-07 16:43:03] [VULKAN_SDL2] Supported codecs are: RGBA UYVY YUYV Y216 Y416 R10k 
[2023-01-07 16:43:03] [video dec.] New incoming video format detected: 1920x1080 @50.00i, codec R10k
[2023-01-07 16:43:03] [video dec.] Successfully reconfigured display to 1920x1080 @50.00i, codec R10k

Intel NUC12 w/Ubuntu 22.04

uv -t testcard:codec=R10k -d vulkan_sdl2

UltraGrid 1.8+ (tags/continuous rev 910a522 built Jan  7 2023 14:24:18)
[2023-01-07 16:53:53] [SDL] Using driver: x11
[2023-01-07 16:53:53] SDL2 initialized successfully.
[2023-01-07 16:53:53] [VULKAN_SDL2] Path to shaders: "/usr/local/bin/../share/ultragrid/vulkan_shaders"
[2023-01-07 16:53:53] [VULKAN_SDL2] yCbCr feature supported.
[2023-01-07 16:53:53] [VULKAN_SDL2] Vulkan uses GPU called: Intel(R) Graphics (ADL GT2)
[2023-01-07 16:53:53] [VULKAN_SDL2] Used Vulkan API: 1.1
[2023-01-07 16:53:53] [VULKAN_SDL2] Recreating swapchain, size: 960x540, format: A2R10G10B10UnormPack32
[2023-01-07 16:53:53] [VULKAN_SDL2] Vulkan display initialised.
[2023-01-07 16:53:53] [VULKAN_SDL2] Recreating render_pipeline
[2023-01-07 16:53:53] [testcard] capture set to 1920x1080 @50.00i, codec R10k, bpp 32, pattern: bars, audio off
[2023-01-07 16:53:53] Control socket listening on port 44791
[2023-01-07 16:53:53] [key control] Stdin is not a TTY - disabling keyboard control.
[2023-01-07 16:53:54] [VULKAN_SDL2] Supported codecs are: RGBA RGB UYVY YUYV Y416 R10k RG48 

[mpiatka]

Hi,

the corruption on Intel should be fixed with 527aa625.

As for the Topton unit, my hypothesis is that it uses a setup similar to gaming laptops where the nvidia gpu renders the frames, which are then presented using the onboard integrated intel gpu to the screen. We tested on several such laptops and all had the color swap issue. This however appears to be a driver issue, because other programs (like mpv with the vulkan backend) exhibit the color swap too. Could you post the output of lspci | grep VGA to verify if the Topton indeed has an Intel gpu? If that is the case, it should be possible to solve the issue by rendering on the Intel using -d vulkan_sdl2:gpu=<id> provided that the intel vulkan drivers are installed.

[alatteri]

HI Martin,

Thanks for doing that.

I'll test the corruption fix this week.

Regarding the Topton, it does have the integrated graphics (i believe the CPU is i9-9300HK) which would make it Intel UHD 630. My goal in testing this unit was the small form factor with Nvidia GPU, to help with decoding 4K SVT-HEVC streams. In recent testing NUC12 can do this using lavd-thread-count=6F, although that does add 6 frames of latency, and SVT has its own additional latency, bringing total 4K SVT-HEVC latency (encode->display) to about 24f@24fps. So I need to test wether using nvdec has a lower latency. If the latency is not less with Nvidia then it doesn't matter anyway as there would be no benefit, NUC12 is smaller, has thunderbolt, stable availability, and reliable manufacture.

Thanks again for everything, I'll post results in a few days.

Alan

[alatteri]

confirmed.... Intel NUC12 now displays 10bit correctly with Vulkan. This is great, thanks.

the corruption on Intel should be fixed with 527aa62.

This also works sort of. The colors are no longer swapped when specifying vulkan_sdl2:gpu=0, but the performance leads to many decode drops. Even when not specifying the GPU, and just letting the NVIDIA do all the decoding/display (with swapped colors), the performance is a little better as using NUC12 with lavd-thread-count=6F. About 23 vs 28 frames of latency). So I'd say it is not worth trying to make this Topton platform work properly. NUC12 is smaller, more reliable, and reputable manufacturer.

As for the Topton unit, my hypothesis is that it uses a setup similar to gaming laptops where the nvidia gpu renders the frames, which are then presented using the onboard integrated intel gpu to the screen. We tested on several such laptops and all had the color swap issue. This however appears to be a driver issue, because other programs (like mpv with the vulkan backend) exhibit the color swap too. Could you post the output of lspci | grep VGA to verify if the Topton indeed has an Intel gpu? If that is the case, it should be possible to solve the issue by rendering on the Intel using -d vulkan_sdl2:gpu=<id> provided that the intel vulkan drivers are installed.