Wrong colors on Android emulator streaming. (API 30)

[tzah4748]

Hi, first of all thanks for this amazing project, its a gem.

Secondly, I am using "scrcpy" to stream Android devices data (Raw H264 NAL Units) through a WebSocket. There is an issue that for some emulators (and maybe devices, haven't checked yet), the picture shows in wrong colors. The emulator below is a Pixel 2 API 30, another device that showed this issue is Pixel C API 30 Same devices with API 28 works fine. All these emulator works fine on the original scrcpy client (that works a bit differently in terms of showing the raw data).

[samirkumardas]

Thank you.

No clue. The actual rendering is done by the browser, so not sure why it is showing the wrong color. Can you provide some portion of the h264 buffer so I can analyze it?

[tzah4748]

https://drive.google.com/drive/folders/1Jw2jY-jyK6jY0vkCOw4DrpJnyPhCa8iI?usp=sharing

Added 2 files there - video and Raw Data, i think the data has been saved as Signed int instead of unsigned int so keep that in mind and convert it.

P.S - The picture looks fine on Firefox browser... it seems like this issue occurs only on Chromium based browsers???

[tzah4748]

Quick update : Setting the 'Force color profile' on chrome://flags will fix this issue! Use the HDR10 (Worked for me)

[samirkumardas]

Interesting is that it played well when I tried the static file https://samirkumardas.github.io/jmuxer/h264_player.html Note: I did not change any Force Color profile flag

As it is playing the static file, it should play the streaming as well. Anyway, I will analyze the packet format.

[tzah4748]

Using the same static file i gave you (When the flag is set to Default)

[samirkumardas]

I am getting fine with the default color profile.

What is your chrome versions and OS?

[tzah4748]

Google Chrome - Version 84.0.4147.125 (Official Build) (64-bit) Windows 10 Pro 64-bit Machine

[samirkumardas]

Same chrome version but on mac OS. Will check on windows

[tzah4748]

I made a quick check on Mac OS Works fine, it seems this issue is related to how Chrome behaves on Windows

[NilStack]

Hi, @tzah4748 and @samirkumardas , I'm also " using scrcpy to stream Android devices data (Raw H264 NAL Units) through a WebSocket." . But always got error "Input object must have video and/or audio property. Make sure it is a valid typed array" and another error "jmuxer.js?b637:220 Error occured while appending video buffer - InvalidStateError: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source.". Do you process raw H264 data before feed it to jmuxer? I just feed it directly. Here is some snippets with jmuxer.

` jmuxer = new JMuxer({ node: "player", mode: "video", flushingTime: 1, clearBuffer: true, debug: true, });

this.wsSocket = new WebSocket("ws://localhost:9999"); this.wsSocket.binaryType = "arraybuffer"; this.wsSocket.addEventListener("message", this.feed);

feed(event) { let data = new Uint8Array(event.data); jmuxer.feed({ video: data, }); }, `

[samirkumardas]

@NilStack it seems the provided h264 data is not valid Annex-B format. Make sure you don't have any additional header data with this.

I would suggest grabbing a portion of h264 buffer and try playing it with this player https://samirkumardas.github.io/jmuxer/h264_player.html

If you share the h264 data, I can examine it as well.

[tzah4748]

@NilStack - You actually need to process the data coming from the scrcpy yes. Look at the scrcpy client code to understand what you actually need to do, there is a bunch of header data coming first and after it each NALU is preceded by a message of its size.

[NilStack]

@tzah4748 Thanks for your reply！I did not send scrcpy header data like one byte 00 code, and other 68 bytes including device name, screen width and screen height. Only NALU is sent by websocket and feed to jmuxer. But h264 data from scrcpy is not Annex-B format, right? Do you make them Annex-B before feeding them to jmuxer? If so, how do you do this? Thanks for your reply again.

@samirkumardas Thanks! I will go with your suggestion.

[tzah4748]

@NilStack - Sorry for the late reply, didn't see your question.

I am actually parsing the NALU with jCodec (on my Java application) to separate the data into single NALU, you can use it to manipulate the incoming raw data from the scrcpy.