selkies-project / selkies-gstreamer

Open-Source Low-Latency Accelerated Linux WebRTC HTML5 Remote Desktop Streaming Platform for Self-Hosting, Containers, Kubernetes, or Cloud/HPC
https://selkies-project.github.io/selkies-gstreamer/
Mozilla Public License 2.0
380 stars 48 forks source link

Color is slightly incorrect from the GStreamer video converter #160

Open ehfd opened 5 months ago

ehfd commented 5 months ago

Relevant (more information about the issue): https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6902 https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1260 https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1315 https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1339 https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2948 https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2991 https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3495 https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3556

There is an incorrect color issue (reproduced in the above issues) where it is concentrated on the video converter.

Other than the video converter, color is consistent between the source (videotestsrc or ximagesrc so no issue here), the encoder (consistent color when the same video converter is used), and output (file and WebRTC video are also consistent color).

cudaconvert and videoconvert are similarly wrong for the colors, but show slightly different ways of being wrong.

Therefore, this is an upstream color converter issue that needs to be fixed in GStreamer.

I think the direct cause is https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3495.

If this is not fixed, users will have a noticeable feeling of degradation of the remote screen's color reproduction.

At least in flat, single-color, static regions, the reproduction should not deviate outside +/- 1 for each RGB color and converge to the exact color when some time passes in a static state. Thus, luma should be independent of chroma changes.