Closed basilgello closed 2 years ago
Firefox running HTML5 client:
aac+mpeg4
: clicking 'unmute` button on Xpra HTML5 client pane spews an error:
Invalid audio codec 'unknown' (expected aac+mpeg4), stopping audio stream Utilities.js:1:521
cerror https://localhost:14500/js/Utilities.js:1
cerror https://localhost:14500/js/Client.js:1
error https://localhost:14500/js/Client.js:1
_process_sound_data https://localhost:14500/js/Client.js:1
_route_packet https://localhost:14500/js/Client.js:1
open https://localhost:14500/js/Protocol.js:1
(Async: EventListener.handleEvent)
open https://localhost:14500/js/Protocol.js:1
open_protocol https://localhost:14500/js/Client.js:1
_do_connect https://localhost:14500/js/Client.js:1
connect https://localhost:14500/js/Client.js:1
(Async: EventListener.handleEvent)
connect https://localhost:14500/js/Client.js:1
init_page https://localhost:14500/index.html:1222
onload https://localhost:14500/index.html:1299
(Async: EventHandlerNonNull)
<anonymous> https://localhost:14500/index.html:1291
jQuery 13
while server log shows Gstreamer lacks compliance
property:
2021-08-09 09:53:49,062 audio capture Error setting up the sound pipeline:
2021-08-09 09:53:49,063 audio capture gst_parse_error: no compliance «compliance» property in element «avenc_aac0» (2)
2021-08-09 09:53:49,063 audio capture GStreamer pipeline for aac+mpeg4:
2021-08-09 09:53:49,063 audio capture pulsesrc device="Xpra-Speaker.monitor" name="src" ! \
2021-08-09 09:53:49,063 audio capture queue name=queue min-threshold-time=0 max-size-buffers=0 max-size-bytes=0 max-size-time=50000000000000 leaky=2 ! \
2021-08-09 09:53:49,063 audio capture volume name=volume volume=1.0 ! \
2021-08-09 09:53:49,063 audio capture avenc_aac compliance=1 perfect-timestamp=1 ! \
2021-08-09 09:53:49,063 audio capture mp4mux faststart=1 streamable=1 fragment-duration=20 presentation-time=0 ! \
2021-08-09 09:53:49,063 audio capture appsink name=sink emit-signals=true max-buffers=10 drop=true sync=false async=false qos=false```
mpeg4+mp3
: the following HTML5 client error
audio loadstart MediaSourceUtil.js:1:4849
audio source open: open MediaSourceUtil.js:1:4390
audio media source open Utilities.js:1:648
using audio codec string for mp3+mpeg4: audio/mp4; codecs="mp3" Utilities.js:1:648
audio: requesting mp3+mpeg4 stream from the server Utilities.js:1:648
broadway decoder initialized Utilities.js:1:648
audio-state: playing Utilities.js:1:648
audio loadedmetadata MediaSourceUtil.js:1:4849
Media resource blob:https://localhost:14500/155def2c-320c-47e8-aba5-d7f93d0b5dff could not be decoded. index.html
audio error MediaSourceUtil.js:1:4849
with Xpra server error:
2021-08-09 10:01:02,447 using pulseaudio device:
2021-08-09 10:01:02,447 'Monitor of Xpra Speaker'
2021-08-09 10:01:02,801 Error playing new-stream bell sound:
2021-08-09 10:01:02,801 [Errno 2] Немає такого файла або каталогу: 'gst-launch-1.0'
2021-08-09 10:01:03,139 client 2 broadway decoder initialized
2021-08-09 10:01:03,140 client 2 audio-state: playing
2021-08-09 10:01:03,141 client 2 audio error: [object MediaError]
2021-08-09 10:01:03,142 client 2 DECODE: error occurred when decoding
2021-08-09 10:01:03,146 client 2 audio: stopping stream
2021-08-09 10:01:03,146 client 2 close_audio_mediasource: audio_source_buffer=[object SourceBuffer], media_source=[object MediaSource], audio=[object HTMLAudioElement]
2021-08-09 10:01:03,146 client 2 audio-state: stopped
2021-08-09 10:01:03,151 audio capture stopping
2021-08-09 10:01:03,426 audio capture using 'mp3' audio codec
2021-08-09 10:01:03,658 client 2 received end-of-stream from server
legacy wav
: the following HTML5 client error:
Array [ "sound data error" ]
0: "sound data error"
length: 1
<prototype>: Array []
Utilities.js:1:521
cerror https://localhost:14500/js/Utilities.js:1
cerror https://localhost:14500/js/Client.js:1
exc https://localhost:14500/js/Client.js:1
_process_sound_data https://localhost:14500/js/Client.js:1
_route_packet https://localhost:14500/js/Client.js:1
open https://localhost:14500/js/Protocol.js:1
(Async: EventListener.handleEvent)
open https://localhost:14500/js/Protocol.js:1
open_protocol https://localhost:14500/js/Client.js:1
_do_connect https://localhost:14500/js/Client.js:1
connect https://localhost:14500/js/Client.js:1
(Async: EventListener.handleEvent)
connect https://localhost:14500/js/Client.js:1
init_page https://localhost:14500/index.html:1222
onload https://localhost:14500/index.html:1299
(Async: EventHandlerNonNull)
<anonymous> https://localhost:14500/index.html:1291
jQuery 13
and Xpra server error:
2021-08-09 10:04:29,976 Error playing new-stream bell sound:
2021-08-09 10:04:29,977 [Errno 2] Немає такого файла або каталогу: 'gst-launch-1.0'
2021-08-09 10:04:29,995 client 3 audio-state: playing
2021-08-09 10:04:29,999 client 3 audio-state: error
2021-08-09 10:04:29,999 client 3 AVBuffer@https://localhost:14500/js/lib/aurora/aurora.js:1:12735
2021-08-09 10:04:29,999 client 3 XpraSource.prototype._on_data@https://localhost:14500/js/lib/aurora/aurora-xpra.js:1:531
2021-08-09 10:04:29,999 client 3 XpraClient.prototype.push_audio_buffer@https://localhost:14500/js/Client.js:1:73974
2021-08-09 10:04:29,999 client 3 XpraClient.prototype.add_sound_data@https://localhost:14500/js/Client.js:1:72013
2021-08-09 10:04:29,999 client 3 XpraClient.prototype._process_sound_data@https://localhost:14500/js/Client.js:1:70573
2021-08-09 10:04:29,999 client 3 XpraClient.prototype._route_packet@https://localhost:14500/js/Client.js:1:11200
2021-08-09 10:04:29,999 client 3 XpraProtocolWorkerHost.prototype.open/<@https://localhost:14500/js/Protocol.js:1:733
2021-08-09 10:04:30,000 client 3 EventListener.handleEvent*XpraProtocolWorkerHost.prototype.open@https://localhost:14500/js/Protocol.js:1:582
2021-08-09 10:04:30,000 client 3 XpraClient.prototype.open_protocol@https://localhost:14500/js/Client.js:1:9540
2021-08-09 10:04:30,000 client 3 XpraClient.prototype._do_connect@https://localhost:14500/js/Client.js:1:9222
2021-08-09 10:04:30,000 client 3 XpraClient.prototype.connect/<@https://localhost:14500/js/Client.js:1:8703
2021-08-09 10:04:30,000 client 3 EventListener.handleEvent*XpraClient.prototype.connect@https://localhost:14500/js/Client.js:1:8587
2021-08-09 10:04:30,000 client 3 init_page@https://localhost:14500/index.html:1222:12
2021-08-09 10:04:30,000 client 3 xhr.onload@https://localhost:14500/index.html:1299:6
2021-08-09 10:04:30,000 client 3 EventHandlerNonNull*@https://localhost:14500/index.html:1291:5
2021-08-09 10:04:30,000 client 3 mightThrow@https://localhost:14500/js/lib/jquery.js:3760:29
2021-08-09 10:04:30,000 client 3 resolve/</process<@https://localhost:14500/js/lib/jquery.js:3828:12
2021-08-09 10:04:30,000 client 3 setTimeout handler*resolve/<@https://localhost:14500/js/lib/jquery.js:3866:16
2021-08-09 10:04:30,000 client 3 fire@https://localhost:14500/js/lib/jquery.js:3494:31
2021-08-09 10:04:30,000 client 3 fireWith@https://localhost:14500/js/lib/jquery.js:3624:7
2021-08-09 10:04:30,000 client 3 fire@https://localhost:14500/js/lib/jquery.js:3632:10
2021-08-09 10:04:30,000 client 3 fire@https://localhost:14500/js/lib/jquery.js:3494:31
2021-08-09 10:04:30,000 client 3 fireWith@https://localhost:14500/js/lib/jquery.js:3624:7
2021-08-09 10:04:30,000 client 3 ready@https://localhost:14500/js/lib/jquery.js:4104:13
2021-08-09 10:04:30,000 client 3 completed@https://localhost:14500/js/lib/jquery.js:4114:9
2021-08-09 10:04:30,001 client 3 EventListener.handleEvent*@https://localhost:14500/js/lib/jquery.js:4130:11
2021-08-09 10:04:30,001 client 3 @https://localhost:14500/js/lib/jquery.js:34:10
2021-08-09 10:04:30,001 client 3 @https://localhost:14500/js/lib/jquery.js:38:4
2021-08-09 10:04:30,001 client 3 audio: stopping stream
2021-08-09 10:04:30,001 client 3 audio-state: stopped
2021-08-09 10:04:30,005 audio capture stopping
2021-08-09 10:04:30,007 client 3 received end-of-stream from server
2021-08-09 10:04:30,008 client 3 audio: stopping stream
2021-08-09 10:04:30,452 audio capture using 'wav' audio codec
http stream: mp3
: the unmute button stays but the HTML5 client spews error:
GEThttps://localhost:14500/audio.mp3?uuid=5cd196ec-7035-afff-6581-5ff2313b22aa
[HTTP/1.0 404 Path not found 48ms]
HTTP load failed with status 404. Load of media resource https://localhost:14500/audio.mp3?uuid=5cd196ec-7035-afff-6581-5ff2313b22aa failed. index.html All candidate resources failed to load. Media load paused.
while nothing criminal in Xpra server log:
2021-08-09 10:07:16,427 client 4 speaker icon clicked, audio_enabled= true 2021-08-09 10:07:16,427 client 4 audio-state: waiting 2021-08-09 10:07:16,427 client 4 starting http stream from https://localhost:14500/audio.mp3?uuid=5cd196ec-7035-afff-6581-5ff2313b22aa
Dropping this line:
https://github.com/Xpra-org/xpra/blob/master/xpra/sound/gstreamer_util.py#L159
fixes aac+mpeg4
for me. Sound quality is definitely not the best but no underruns on GTK3 client.
opus
: smooth sound playback
As per https://github.com/Xpra-org/xpra/blob/8d5ba7a8b8f8937dcdc3df5274aa3f2e6035ca9f/xpra/sound/gstreamer_util.py#L211-L216
opus
should be the default if all the dependencies are installed correctly.
I have re-ordered the list based on your findings - since I am unable to test properly "thanks" to pipewire.
I do get opus
automagically when I just start the client:
audio playback pipeline elements=['appsrc name=src emit-signals=0 \
block=0 is-live=0 stream-type=0 format=4', \
'oggdemux', 'opusdec', 'audioconvert', 'audioresample', \
'queue name=queue min-threshold-time=0 max-size-buffers=0 max-size-bytes=0 max-size-time=450000000 leaky=2', \
'volume name=volume volume=0', \
'pulsesink sync=False async=True qos=True client-name="Xpra" name="sink"']
But you're saying that:
vorbis+mka
(aka default): choppy sound, often overruns
So your package is broken somehow.
The packages from xpra.org
definitely don't do that.
aac+mpeg4
(..) Dropping this line (..) fixes aac+mpeg4 for me
Done, thanks!
mpeg4+mp3
: the following HTML5 client error (..)Media resource blob:https://localhost:14500/155def2c-320c-47e8-aba5-d7f93d0b5dff could not be decoded.
Ah. Those errors are completely undecipherable for me.
mpeg4+mp3
support in the browser requires very specific settings on the server side.
When it fails, the feedback you get from the browser is error
or could not be decoded
.
Very difficult to fix reliably.
This used to work, at some point, on some browsers...
We also have a blacklist of combinations that don't work. It may also need updating.
with Xpra server error:
That's just the html5 client error being forwarded to the server.
legacy wav
: the following HTML5 client error:
Could just be rencodeplus
giving bytes
instead of strings.
I'll have to find a way to try it somehow.
http stream
: mp3: the unmute button stays but the HTML5 client spews error
Support for this has been removed from the xpra server a while back, and now also from the html5 client: https://github.com/Xpra-org/xpra-html5/commit/40aa975420e494ac5c8f79bdf88d4c3c77e577f1 (it was only really useful for some versions of IE - if you could tolerate the 5 second audio delay..)
Ah. Those errors are completely undecipherable for me.
Easier than we both thought :rofl: Proxy server does not forward ttps://localhost:14500/<uuid>
.
Audio works with HTML5 client running like this:
xpra start --start='google-chrome --incognito' --speaker=on --bind-tcp=0.0.0.0:10000 --daemon=no
and navigating to http://localhost:10000
in Firefox
I'll have to find a way to try it somehow.
If you want I can share my Podman containerfilles! Debian+xpra from your repo or my builds (I can supply them, too, since they are not yet merged into Debian by Dmitry).
Also, I guess invoking HTML5 client with audio-forwarding
enabled should instruct Xpra proxy to start Xpra seamless / desktop / shadow server with --speaker=on
(and another handle should do the same for --microphone=on
). And this should have no effect on HTML5 connect
action.
Also, I guess https://xxx/<uuid>
must be protected by authentication somehow so an attacker with UUID bruteforce can not hijack the audio stream,
Proxy server does not forward
I don't understand.
If you want I can share my Podman...
I can run in containers or virtual machines, I just don't like doing it when developing or debugging things.
should instruct Xpra proxy to start Xpra seamless / desktop / shadow server with
--speaker=on
The default is enabled so the proxy doesn't need to do anything.
Also, I guess
https://xxx/<uuid>
must be protected...
It's virtually impossible to brute force, the uuid has too many bits. In any case, that's now removed.
I don't understand.
Easy :) Xpra server directly accessible handles http(s)://host:port/uuid
by serving a media stream with proper mimetype. Xpra proxy server serves index.html
contents with text/html
mimetype. Media DOM element expects sound data stream, not text of index.html
.
In any case, that's now removed
Wait, do you mean aac+mpeg4
or opus
do not use uuid scheme at all? If it does not, then I have to investigate why directly accessible Xpra server listening on random TCP port delivers sound packets properly, while proxied server does not
Wait, do you mean
aac+mpeg4
oropus
do not use uuid scheme at all?
Correct. The only http stream was an mp3 one that was only useful for Internet Explorer and this has now been removed.
Audio streams are handled in a similar way to the python client: audio packets are received through the websocket connection and forwarded to either aurora (javascript decoding) or the native MediaSource
decoder.
As for why it doesn't work through the proxy: it could well be something rencodeplus
related there.
As for why it doesn't work through the proxy: it could well be something rencodeplus related there.
HTML5 client never worked for me with python3-rencode
, too, so it must be something different. I have to check Xpra GTK3 client connected through proxy instance niw.
I have to check Xpra GTK3 client connected through proxy instance niw.
This, surprisingly, works!
However, no HTML5 client plays audio if proxy server is in use: server.log
legacy: wav
does not work via proxy because second packet in a sequence (all zeros 0x00) pushed by _on_data
as-is and AVBuffer function does not know what type the input is.
Why is this different when going through the proxy? does the proxy strip some attributes that would tell the aurora decoder what type of packet it is?
Are we meant to set this.audio_aurora_ctx.format
?
More importantly. how did you find that out?
More importantly. how did you find that out?
For now, I have debugged only missing buffer for legacy wav
.
xpra start --start=mpv --speaker=on --bind-tcp=0.0.0.0:10000
xpra start --start=mpv --speaker=on
/usr/share/xpra/www/js/aurora.js
minified by Debian package with unminified one from source tree and delete correspondent *.gz
and *.br
files so they dont interferehttps://localhost:14500
using Google Chrome developer modeinput
argument as valid Uint8Array
Uint8Array
equal to first sound-data packet, but second call's input
is \u0000\u0000\u0000…
(1746 times) yielding an error https://github.com/Xpra-org/xpra-html5/blob/master/html5/js/lib/aurora/aurora.js#L629Looks the issue is decoded sound packet passed through proxy. setting packet-encoders
to bencode
does not help either. At least, not with wav
I debugged the issue!
Since wav
is especially hard to troubleshoot, I decided to switch to MediaSource
codecs, like aac+mpeg4
or webm+opus
.
First of all, I applied the following patch to Client.js
:
diff --git a/html5/js/Client.js b/html5/js/Client.js
index 5728faf..2dc86ce 100644
--- a/html5/js/Client.js
+++ b/html5/js/Client.js
@@ -89,6 +89,7 @@ XpraClient.prototype.init_settings = function(container) {
XpraClient.prototype.init_state = function(container) {
// state
+ this.audio_packets = 0;
this.connected = false;
this.desktop_width = 0;
this.desktop_height = 0;
@@ -3133,6 +3134,7 @@ XpraClient.prototype._process_sound_data = function(packet, ctx) {
}
catch(e) {
ctx.on_audio_state_change("error", ""+e);
+ ctx.clog(e, "sound data error")
ctx.exc(e, "sound data error");
ctx.close_audio();
}
@@ -3160,6 +3162,7 @@ XpraClient.prototype.add_sound_data = function(codec, buf, metadata) {
//push metadata first:
for (let i = 0; i < metadata.length; i++) {
this.debug("audio", "metadata[", i, "]=", metadata[i], ", length=", metadata[i].length, ", type=", Object.prototype.toString.call(metadata[i]));
+ this.clog("metadata[", i, "]=", metadata[i], ", length=", metadata[i].length, ", type=", Object.prototype.toString.call(metadata[i]));
this.audio_buffers.push(Utilities.StringToUint8(metadata[i]));
}
//since we have the metadata, we should be good to go:
@@ -3205,6 +3209,7 @@ XpraClient.prototype.add_sound_data = function(codec, buf, metadata) {
};
XpraClient.prototype._audio_start_stream = function() {
+ this.audio_packets = 0;
this.debug("audio", "audio start of "+this.audio_framework+" "+this.audio_codec+" stream");
if (this.audio_state=="playing" || this.audio_state=="waiting") {
//nothing to do: ready to play
@@ -3256,6 +3261,10 @@ XpraClient.prototype._audio_ready = function() {
XpraClient.prototype.push_audio_buffer = function(buf) {
if (this.audio_framework=="mediasource") {
this.audio_source_buffer.appendBuffer(buf);
+ if (this.audio_packets <= 5) {
+ this.clog("appendBuffer size=", buf.length, "buf=", Utilities.ArrayBufferToBase64(buf));
+ this.audio_packets++;
+ }
const b = this.audio_source_buffer.buffered;
if (b && b.length>=1) {
//for (let i=0; i<b.length;i++) {
The patch just prints metadata and Base64-encoded buffer supplied to appendBuffer
, which is later decoded by FFmpeg's DecodeRawMediaPacket
.
The good first aac+mpeg4
packet dumped from the HTML5 client directly ttached to Xpra server has the MPEG4-TS headers wrapping the AAC ADTS stream info:
AAAAIGZ0eXBtcDQyAAAAAG1wNDJtcDQxaXNvbWlzbzIAAALJbW9vdgAAAGxtdmhkAAAAAN04D8LdOA/CAAAHCAAAAAAAAQAAAQAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAdx0cmFrAAAAXHRraGQAAAAH3TgPwt04D8IAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAE7bWRpYQAAACBtZGhkAAAAAN04D8LdOA/CAACsRAAAAABVxAAAAAAALWhkbHIAAAAAAAAAAHNvdW4AAAAAAAAAAAAAAABTb3VuZEhhbmRsZXIAAAAA5m1pbmYAAAAQc21oZAAAAAAAAAAAAAAAJGRpbmYAAAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAAAqnN0YmwAAABec3RzZAAAAAAAAAABAAAATm1wNGEAAAAAAAAAAQAAAAAAAAAAAAIAEAAAAACsRAAAAAAAKmVzZHMAAAAAAxwAAQAEFEAVAAAAAAAAAAAAAAAFBRIQVuUABgECAAAAEHN0dHMAAAAAAAAAAAAAABBzdHNjAAAAAAAAAAAAAAAUc3RzegAAAAAAAAAAAAAAAAAAABBzdGNvAAAAAAAAAAAAAAA9dWR0YQAAADVtZXRhAAAAAAAAACFoZGxyAAAAAG1obHJtZGlyAAAAAAAAAAAAAAAAAAAAAAhpbHN0AAAAPXVkdGEAAAA1bWV0YQAAAAAAAAAhaGRscgAAAABtaGxybWRpcgAAAAAAAAAAAAAAAAAAAAAIaWxzdAAAADxtdmV4AAAAFG1laGQBAAAAAAAAAAAAAAAAAAAgdHJleAAAAAAAAAABAAAAAQAAAAAAAAAAAAAAAAAAAGBtb29mAAAAEG1maGQAAAAAAAAAAQAAAEh0cmFmAAAAHHRmaGQAAAA4AAAAAQAABAAAAAEWAAAAwAAAABB0ZmR0AAAAAAAAAAAAAAAUdHJ1bgAAAAEAAAABAAAAaAAAAR5tZGF03gIATGF2YzU4LjkxLjEwMABCVambC2EaRX9fu3GT48U9/XXxUJm7cC2VqquJljkb7L5T5N86N86dj9/Vt02e2YmGzoqxFlU5UP5bHyjKeE7+qIAoQ+yWLmzlecskhs5ZQMTeewAybNVVRujKIRTVc5VUYSIynjLAAAAD0gOf3BIAS5n1AAAjjD23yqaQ+AZh5Yi3gW4gbrvXs+0OjjqB/LJYM87EYePt+HliG0nzDh6Ri3gPwDtwDlPXRrjMJxK1wYiPh9FIjd8iyh+oPAHB9RSRnBx2b+77VXDeAE4dl42p+e79AKqAmrCsTT8H8fFSVmvPdPf38/FQmbtpMlVmqq6SAKvOve5AJa0WAAAAAAGQAAACQjw=
Decoding it with Base64 and feeding it to ffprobe
shows no errors:
$ base64 -d aac+mpeg4-good01.txt | ffprobe -loglevel repeat+level+trace -
[info] ffprobe version 4.3.2-0+deb11u2 Copyright (c) 2007-2021 the FFmpeg developers
[info] built with gcc 10 (Debian 10.2.1-6)
... snip ...
[trace] Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:1127
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [debug] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'ftyp' parent:'root' sz: 32 8 9223372036854775807
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [debug] ISO: File Type Major Brand: mp42
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'moov' parent:'root' sz: 713 40 9223372036854775807
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'mvhd' parent:'moov' sz: 108 8 705
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] time scale = 1800
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'trak' parent:'moov' sz: 476 116 705
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'tkhd' parent:'trak' sz: 92 8 468
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'mdia' parent:'trak' sz: 315 100 468
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'mdhd' parent:'mdia' sz: 32 8 307
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'hdlr' parent:'mdia' sz: 45 40 307
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] ctype=[0][0][0][0]
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] stype=soun
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'minf' parent:'mdia' sz: 230 85 307
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'smhd' parent:'minf' sz: 16 8 222
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'dinf' parent:'minf' sz: 36 24 222
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'dref' parent:'dinf' sz: 28 8 28
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [debug] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'stbl' parent:'minf' sz: 170 60 222
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'stsd' parent:'stbl' sz: 94 8 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] size=78 4CC=mp4a codec_type=1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] audio channels 2
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] version =0, isom =1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'esds' parent:'stsd' sz: 42 8 42
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] MPEG-4 description: tag=0x03 len=28
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] MPEG-4 description: tag=0x04 len=20
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] esds object type id 0x40
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] MPEG-4 description: tag=0x05 len=5
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] Specific MPEG-4 header len=5
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] mp4a config channels 2 obj 2 ext obj 5 sample rate 44100 ext sample rate 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'stts' parent:'stbl' sz: 16 102 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] track[0].stts.entries = 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'stsc' parent:'stbl' sz: 16 118 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] track[0].stsc.entries = 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'stsz' parent:'stbl' sz: 20 134 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] sample_size = 0 sample_count = 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'stco' parent:'stbl' sz: 16 154 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'udta' parent:'trak' sz: 61 415 468
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'meta' parent:'udta' sz: 53 8 53
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'hdlr' parent:'meta' sz: 33 8 41
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] ctype=mhlr
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] stype=mdir
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'ilst' parent:'meta' sz: 8 41 41
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'udta' parent:'moov' sz: 61 592 705
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'meta' parent:'udta' sz: 53 8 53
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'hdlr' parent:'meta' sz: 33 8 41
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] ctype=mhlr
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] stype=mdir
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'ilst' parent:'meta' sz: 8 41 41
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'mvex' parent:'moov' sz: 60 653 705
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'mehd' parent:'mvex' sz: 20 8 52
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'trex' parent:'mvex' sz: 32 28 52
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'moof' parent:'root' sz: 96 753 9223372036854775807
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] moof offset 2e9
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'mfhd' parent:'moof' sz: 16 8 88
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'traf' parent:'moof' sz: 72 24 88
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'tfhd' parent:'traf' sz: 28 8 64
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] frag flags 0xc0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'tfdt' parent:'traf' sz: 16 36 64
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'trun' parent:'traf' sz: 20 52 64
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] flags 0x1 entries 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [debug] found tfdt time 0, using it for dts
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] first sample flags 0xc0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] AVIndex stream 0, sample 1, offset 351, dts 0, size 278, distance 0, keyframe 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] type:'mdat' parent:'root' sz: 286 849 9223372036854775807
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] on_parse_exit_offset=849
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [debug] Before avformat_find_stream_info() pos: 849 bytes read:1127 seeks:0 nb_streams:1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] stream 0, sample 0, dts 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [debug] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] stream 0: start_time: 0 duration: 0.02322
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [trace] format: start_time: 0 duration: 0.02322 (estimate from stream) bitrate=0 kb/s
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556798307d80] [debug] After avformat_find_stream_info() pos: 1127 bytes read:1127 seeks:0 frames:1
[info] Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'pipe:':
[info] Metadata:
[info] major_brand : mp42
[info] minor_version : 0
[info] compatible_brands: mp42mp41isomiso2
[info] creation_time : 2021-08-10T11:13:06.000000Z
[info] Duration: 00:00:00.02, start: 0.000000, bitrate: N/A
[info] Stream #0:0(und), 1, 1/44100: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 95 kb/s (default)
[info] Metadata:
[info] creation_time : 2021-08-10T11:13:06.000000Z
[info] handler_name : SoundHandler
[AVIOContext @ 0x556798310bc0] [verbose] Statistics: 1127 bytes read, 0 seeks
The first aac+mpeg4
packet from Xpra server beyond Xpra proxy server, however, has valid MPEG4-TS headers but AAC (or OPUS) streams are zeroed out:
AAAAIGZ0eXBtcDQyAAAAAG1wNDJtcDQxaXNvbWlzbzIAAALJbW9vdgAAAGxtdmhkAAAAAN04EKndOBCpAAAHCAAAAAAAAQAAAQAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAdx0cmFrAAAAXHRraGQAAAAH3TgQqd04EKkAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAE7bWRpYQAAACBtZGhkAAAAAN04EKndOBCpAACsRAAAAABVxAAAAAAALWhkbHIAAAAAAAAAAHNvdW4AAAAAAAAAAAAAAABTb3VuZEhhbmRsZXIAAAAA5m1pbmYAAAAQc21oZAAAAAAAAAAAAAAAJGRpbmYAAAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAAAqnN0YmwAAABec3RzZAAAAAAAAAABAAAATm1wNGEAAAAAAAAAAQAAAAAAAAAAAAIAEAAAAACsRAAAAAAAKmVzZHMAAAAAAxwAAQAEFEAVAAAAAAAAAAAAAAAFBRIQVuUABgECAAAAEHN0dHMAAAAAAAAAAAAAABBzdHNjAAAAAAAAAAAAAAAUc3RzegAAAAAAAAAAAAAAAAAAABBzdGNvAAAAAAAAAAAAAAA9dWR0YQAAADVtZXRhAAAAAAAAACFoZGxyAAAAAG1obHJtZGlyAAAAAAAAAAAAAAAAAAAAAAhpbHN0AAAAPXVkdGEAAAA1bWV0YQAAAAAAAAAhaGRscgAAAABtaGxybWRpcgAAAAAAAAAAAAAAAAAAAAAIaWxzdAAAADxtdmV4AAAAFG1laGQBAAAAAAAAAAAAAAAAAAAgdHJleAAAAAAAAAABAAAAAQAAAAAAAAAAAAAAAAAAAGBtb29mAAAAEG1maGQAAAAAAAAAAQAAAEh0cmFmAAAAHHRmaGQAAAA4AAAAAQAABAAAAAGBAAAAwAAAABB0ZmR0AAAAAAAAAAAAAAAUdHJ1bgAAAAEAAAABAAAAaAAAAYltZGF0AAAAAAAAAAUIAAkBAAEAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAAYAAAcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAkAAAAAAAAAAAAAAAAAAAAAAAAGAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
Feeding it to ffprobe
shows an error which in fact is a result of zeroed memory area starting offset 0x351 (849):
$ base64 -d aac+mpeg4-bad01.txt | ffprobe -loglevel repeat+level+trace -
[info] ffprobe version 4.3.2-0+deb11u2 Copyright (c) 2007-2021 the FFmpeg developers
[info] built with gcc 10 (Debian 10.2.1-6)
... snip ...
[pipe @ 0x559c3b348840] [debug] Setting default whitelist 'crypto,data'
[trace] Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:1234
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [debug] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'ftyp' parent:'root' sz: 32 8 9223372036854775807
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [debug] ISO: File Type Major Brand: mp42
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'moov' parent:'root' sz: 713 40 9223372036854775807
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'mvhd' parent:'moov' sz: 108 8 705
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] time scale = 1800
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'trak' parent:'moov' sz: 476 116 705
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'tkhd' parent:'trak' sz: 92 8 468
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'mdia' parent:'trak' sz: 315 100 468
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'mdhd' parent:'mdia' sz: 32 8 307
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'hdlr' parent:'mdia' sz: 45 40 307
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] ctype=[0][0][0][0]
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] stype=soun
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'minf' parent:'mdia' sz: 230 85 307
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'smhd' parent:'minf' sz: 16 8 222
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'dinf' parent:'minf' sz: 36 24 222
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'dref' parent:'dinf' sz: 28 8 28
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [debug] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'stbl' parent:'minf' sz: 170 60 222
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'stsd' parent:'stbl' sz: 94 8 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] size=78 4CC=mp4a codec_type=1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] audio channels 2
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] version =0, isom =1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'esds' parent:'stsd' sz: 42 8 42
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] MPEG-4 description: tag=0x03 len=28
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] MPEG-4 description: tag=0x04 len=20
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] esds object type id 0x40
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] MPEG-4 description: tag=0x05 len=5
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] Specific MPEG-4 header len=5
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] mp4a config channels 2 obj 2 ext obj 5 sample rate 44100 ext sample rate 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'stts' parent:'stbl' sz: 16 102 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] track[0].stts.entries = 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'stsc' parent:'stbl' sz: 16 118 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] track[0].stsc.entries = 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'stsz' parent:'stbl' sz: 20 134 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] sample_size = 0 sample_count = 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'stco' parent:'stbl' sz: 16 154 162
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'udta' parent:'trak' sz: 61 415 468
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'meta' parent:'udta' sz: 53 8 53
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'hdlr' parent:'meta' sz: 33 8 41
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] ctype=mhlr
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] stype=mdir
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'ilst' parent:'meta' sz: 8 41 41
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'udta' parent:'moov' sz: 61 592 705
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'meta' parent:'udta' sz: 53 8 53
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'hdlr' parent:'meta' sz: 33 8 41
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] ctype=mhlr
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] stype=mdir
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'ilst' parent:'meta' sz: 8 41 41
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'mvex' parent:'moov' sz: 60 653 705
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'mehd' parent:'mvex' sz: 20 8 52
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'trex' parent:'mvex' sz: 32 28 52
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'moof' parent:'root' sz: 96 753 9223372036854775807
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] moof offset 2e9
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'mfhd' parent:'moof' sz: 16 8 88
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'traf' parent:'moof' sz: 72 24 88
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'tfhd' parent:'traf' sz: 28 8 64
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] frag flags 0xc0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'tfdt' parent:'traf' sz: 16 36 64
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'trun' parent:'traf' sz: 20 52 64
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] flags 0x1 entries 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [debug] found tfdt time 0, using it for dts
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] first sample flags 0xc0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] AVIndex stream 0, sample 1, offset 351, dts 0, size 385, distance 0, keyframe 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] type:'mdat' parent:'root' sz: 393 849 9223372036854775807
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] on_parse_exit_offset=849
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [debug] Before avformat_find_stream_info() pos: 849 bytes read:1234 seeks:0 nb_streams:1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] stream 0, sample 0, dts 0
[aac @ 0x559c3b349940] [debug] stereo with SCE
[aac @ 0x559c3b349940] [debug] channel element 0.0 duplicate
[aac @ 0x559c3b349940] [error] channel element 0.0 is not allocated
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [debug] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] stream 0: start_time: 0 duration: 0.02322
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [trace] format: start_time: 0 duration: 0.02322 (estimate from stream) bitrate=0 kb/s
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x559c3b347d80] [debug] After avformat_find_stream_info() pos: 1234 bytes read:1234 seeks:0 frames:1
[info] Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'pipe:':
[info] Metadata:
[info] major_brand : mp42
[info] minor_version : 0
[info] compatible_brands: mp42mp41isomiso2
[info] creation_time : 2021-08-10T11:16:57.000000Z
[info] Duration: 00:00:00.02, start: 0.000000, bitrate: N/A
[info] Stream #0:0(und), 1, 1/44100: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 132 kb/s (default)
[info] Metadata:
[info] creation_time : 2021-08-10T11:16:57.000000Z
[info] handler_name : SoundHandler
[AVIOContext @ 0x559c3b350bc0] [verbose] Statistics: 1234 bytes read, 0 seeks
Now the question: why does Xpra proxy server rewrite / transcode the audio stream packets?
Now the question: why does Xpra proxy server rewrite / transcode the audio stream packets?
It isn't meant to - and it probably doesn't do it.
My guess is that it has something to do with large strings getting blanked out by the html5 client whereas Uint8Array
s are copied from the worker without problems. (see how draw
packets are handled).
And this may explain some of the other problems too.
This problem could also be fixed by adding another special case in process_server_packet
:
https://github.com/Xpra-org/xpra/blob/1d1b1a7e3e1f47efa9a867788ef4cf8f59456f63/xpra/server/proxy/proxy_instance.py#L444, calling self._packet_recompress
like the other special cases do.
But the html5 client should be fixed too.
And this may explain some of the other problems too.
Probably you are right because attaching GTK3 client to Xpra proxy like this:
GTK3 - wss -> Xpra proxy (:14500) - socket -> Xpra server
plays sound without issues
Adding self._packet_recompress(packet, 2, "sound-data") did not help: HTML5 client complains on overflows…
The default is enabled so the proxy doesn't need to do anything.
Meh... it is Debian packaging again. I guess when @onlyjob gets well we need to discuss some configuration tweaks introduced by him back in 2013-2016, namely https://salsa.debian.org/debian/xpra/-/blob/master/debian/patches/conf-default-codec-order.patch and https://salsa.debian.org/debian/xpra/-/blob/master/debian/patches/conf-default-speaker.patch
My guess is that it has something to do with large strings getting blanked out by the html5 client whereas Uint8Arrays are copied from the worker without problems. (see how draw packets are handled). And this may explain some of the other problems too.
Now I am going to trace this assumption.
@basilgello how about the fix above for the proxy?
And this should fix the html5 client (for older proxies without the fix above), in 3 commits because I made a complete mess of it (I really need to sort out pipewire support so I can test locally): https://github.com/Xpra-org/xpra-html5/commit/83dc98f88d97523cb06a3253b715f98a289c78d1 + https://github.com/Xpra-org/xpra-html5/commit/226718ba82f524eb7a374b702341c609094ca744 + https://github.com/Xpra-org/xpra-html5/commit/b28eadd3a24a1258012187bb6fe63941aa5aca85
FYI: I made a blooper in the xpra-html5 4.3 release: https://github.com/Xpra-org/xpra-html5/commit/6a2f0b40b7dc33054509f7f222fec36c8d419970
So I intend to release 4.3.1 soon to fix that.
Then I'm going to try to move the decoding to a worker thread. (should really help with h264 and Javascript based pre-processing like rgb24
)
how about the fix above for the proxy?
Do you want me to build trunk Xpra + HTML5 now?
Do you want me to build trunk Xpra + HTML5 now?
When PulseAudio does not die shortly after start-up, it works with HTML5. Now I would like to check why did it start crashing... And not on Bromite for Android (based on Chromium 92).
As for pulseaudio start, starting Xpra server manually like xpra start --speaker=on
starts pulseaudio correctly, but starting it via proxy makes PA terminate with main.c: D-Bus name org.PulseAudio1 already taken
error. On deletion, /run/user/1000/xpra/1/pulse
does not get removed, too.
This is because Xpra proxy does not allocate D-Bus daemon when starting a session with speaker
enabled:
DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus'
while manual start does allocate one:
DBUS_SESSION_BUS_ADDRESS': 'unix:abstract=/tmp/dbus-ZT8nXZrD8a,guid=ff33bdf97324ffdb568294366115661d'
Sound playback works in Firefox browser as client but fails on Bromite for Android.
As for pulseaudio start, starting Xpra server manually like
xpra start --speaker=on
starts pulseaudio correctly, but starting it via proxy makes PA terminate withmain.c: D-Bus name org.PulseAudio1 already taken
error. On deletion,/run/user/1000/xpra/1/pulse
does not get removed, too.This is because Xpra proxy does not allocate D-Bus daemon when starting a session with
speaker
enabled:DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus'
while manual start does allocate one:
DBUS_SESSION_BUS_ADDRESS': 'unix:abstract=/tmp/dbus-ZT8nXZrD8a,guid=ff33bdf97324ffdb568294366115661d'
Sound playback works in Firefox browser as client but fails on Bromite for Android.
Possible fix for D-Bus issue above and #3230 :
diff --git a/xpra/scripts/server.py b/xpra/scripts/server.py
index a6e1133..100c97e 100644
--- a/xpra/scripts/server.py
+++ b/xpra/scripts/server.py
@@ -1181,7 +1181,11 @@ def do_run_server(script_file, cmdline, error_cb, opts, extra_args, mode, displa
except ImportError as e:
dbuslog("dbus components are not installed: %s", e)
else:
- dbus_pid, dbus_env = start_dbus(opts.dbus_launch)
+ # Speaker and microphone forwarding needs private D-Bus and
+ # PulseAudio daemons
+ from xpra.server import server_features
+ always = server_features.audio
+ dbus_pid, dbus_env = start_dbus(opts.dbus_launch, always)
if dbus_env:
dbuslog("started new dbus instance: %s", dbus_env)
save_session_file("dbus.pid", "%s" % dbus_pid)
diff --git a/xpra/server/dbus/dbus_start.py b/xpra/server/dbus/dbus_start.py
index 2f5f920..5527cb3 100755
--- a/xpra/server/dbus/dbus_start.py
+++ b/xpra/server/dbus/dbus_start.py
@@ -15,13 +15,13 @@ from xpra.log import Logger
log = Logger("dbus")
-def start_dbus(dbus_launch):
+def start_dbus(dbus_launch, always=False):
if not dbus_launch or dbus_launch.lower() in FALSE_OPTIONS:
log("start_dbus(%s) disabled", dbus_launch)
return 0, {}
bus_address = os.environ.get("DBUS_SESSION_BUS_ADDRESS")
log("dbus_launch=%r, current DBUS_SESSION_BUS_ADDRESS=%s", dbus_launch, bus_address)
- if bus_address:
+ if bus_address and not always:
log("start_dbus(%s) disabled, found an existing DBUS_SESSION_BUS_ADDRESS=%s", dbus_launch, bus_address)
return 0, {}
assert POSIX
PS: I probably have watched Big Buck Bunny a hundred times already :rofl:
Is your proxy running as root?
Perhaps we should always start dbus and discard the DBUS_SESSION_BUS_ADDRESS
.
It could just be remnant for when people were starting xpra via dbus-launch
.
I can't see a valid reason for re-using an existing dbus session.
Is your proxy running as root?
Yes, Debian packaging runs it as root.
Perhaps we should always start dbus and discard the DBUS_SESSION_BUS_ADDRESS.
Maybe, except of shadow
servers maybe ?
How about 65c8f1c903a3afb7adec871c6909de7acb7895fa ?
Maybe, except of
shadow
servers maybe ?
The whole dbus section already has this:
if not shadowing and POSIX and not OSX:
Did you check if either or both of the fixes above worked? (as per https://github.com/Xpra-org/xpra/issues/3234#issuecomment-897693214)
Did you check if either or both of the fixes above worked?
If by "fixes above" you mean my patch - I never publush things that don't work for me :)
65c8f1c shoukd be good to go, too :)
freshly compiled trunk: the dbus + pulseaudio servers are spawned as expected :)
If by "fixes above" you mean my patch
No. I meant the fixes I linked to previously. Specifically:
I would expect either set of fixes to resolve your sound issues with the html5 client when going through the proxy:
Yes, that worked, thanks! Issues include broken playback on Bromite for Android but it is out of scope since Firefox / Chrome play sound fine (but ~5-6 second delay)
I don't see any issues left here, let's close and open new tickets as needed.
Testing the following setup:
Firefox -> Xpra server with private PulseAudio server -> GTK3 client
with various speaker codecs:
Attaching to Xpra server with every possible codec option gives:
vorbis+mka
(aka default): choppy sound, often overrunsvorbis
: no sound at all, sound level on graph is zeroflac
: smooth sound playbackflag+ogg
: quieter than plainflac
, minor underrunswav+lz4
: smooth sound playbackwav+lzo
: quieter thanwav+lz4
, minor underrunswav
: smooth sound playbackwavpack
: smooth sound playbackspeex+ogg
: low-quality sound, but without chopsmp3
: medium quality sound with minor overrunsaac+mpeg4
: no sound at all (!!!)opus+ogg
: low quality sound, with noticeable underrunsopus
: smooth sound playbackopus+mka
: low quality sound, with noticeable underruns