Closed aler9 closed 4 weeks ago
Hi @aler9
Thank you so much for your amazing work and findings.
I'm testing new behave
My origin file is from Surround sound test files in (almost) every format , Google Drive folder
I run this command tu publish srt (setting b-frames to 0 and transcode to opus)
ffmpeg -re -stream_loop 100 -i "/home/dito/Downloads/AAC 5.1.mp4" -c:a libopus -c:v libx264 -bf 0 -f mpegts "srt://localhost:8890?streamid=publish:mystream"
Stream #0:0(und): Video: h264, yuv420p(progressive), 800x600, q=2-31, 8 fps, 90k tbn (default)
Metadata:
creation_time : 2012-05-17T00:54:33.000000Z
handler_name : GPAC ISO Video Handler
vendor_id : [0][0][0][0]
encoder : Lavc60.31.102 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Stream #0:1(und): Audio: opus, 48000 Hz, 5.1, flt, 320 kb/s (default)
Metadata:
creation_time : 2012-05-17T00:56:13.000000Z
handler_name : GPAC ISO Audio Handler
vendor_id : [0][0][0][0]
encoder : Lavc60.31.102 libopus
Receiver
ffplay -rtsp_transport tcp "rtsp://localhost:8554/mystream"
Stream #0:0: Video: h264 (High), yuv420p(progressive), 800x600, 8 fps, 16 tbr, 90k tbn
Stream #0:1: Audio: none, 48000 Hz, 5.1
MtxServer logs
2024/05/20 18:03:24 INF [SRT] [conn 127.0.0.1:49250] opened
2024/05/20 18:03:24 INF [SRT] [conn 127.0.0.1:49250] is reading from path 'mystream', 2 tracks (H264, Opus)
No audio
ffplay "srt://localhost:8890?streamid=read:mystream"
Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), none, 90k tbr, 90k tbn
Stream #0:1[0x101]: Audio: opus (Opus / 0x7375704F), 48000 Hz, 5.1, fltp
Audio works with srt recewiver
MediaMtx
2024/05/20 18:03:49 WAR [SRT] [conn 127.0.0.1:41095] frame is too big
WebRtc http://localhost:8889/mystream/
Audio works, but sound is demaged (even thought you can hear unclean some words and signals)
2024/05/20 18:06:16 INF [SRT] [conn 127.0.0.1:54250] is publishing to path 'mystream', 2 tracks (H264, Opus)
2024/05/20 18:06:33 INF [WebRTC] [session 0de41c5f] created by [::1]:43468
2024/05/20 18:06:33 INF [WebRTC] [session 0de41c5f] peer connection established, local candidate: host/udp/172.19.0.1/8189, remote candidate: prflx/udp/192.168.2.9/39560
2024/05/20 18:06:33 INF [WebRTC] [session 0de41c5f] is reading from path 'mystream', 2 tracks (H264, Opus)
2024/05/20 18:06:36 WAR [SRT] [conn 127.0.0.1:54250] frame is too big
2024/05/20 18:06:40 WAR [SRT] [conn 127.0.0.1:54250] frame is too big
2024/05/20 18:06:42 WAR [SRT] [conn 127.0.0.1:54250] frame is too big
Hello @ditogam, for your tests did you use the latest release (v1.8.2) or a nightly binary that includes #3375?
Anyway, streaming multichannel Opus with WebRTC is still a work-in-progress and #3375 is yet to be fully tested and merged, in particular, the error you reported
2024/05/20 18:03:49 WAR [SRT] [conn 127.0.0.1:41095] frame is too big
is not covered by the patch yet and is due to an excessive size of the audio packets. I'll see what i can do about it before merging the patch.
Hi @aler9 .
I tested against PR 3375, feature/webrtc-multiopus
branch cheechout and run from GoLand
about frame is too big
I just included in logs
hello @ditogam, i found a bug in parsing timestamps of Opus packets coming from SRT and i fixed it (#3388). This should fix the intermittent audio issue on Chrome.
Regarding the "frame is too big issue", from my tests it happens when bitrate is very high. High bitrate Opus tracks can be streamed with SRT indeed, but they cannot be streamed with WebRTC or RTSP since the specification tells that "An RTP payload MUST contain exactly one Opus packet as defined by that document.", therefore the maximum Opus packet size must be lower than the maximum RTP packet size (which is the UDP MTU). Fortunately the issue only happens with unreasonably high bitrates (higher than 512k), therefore it should be enough to set an explicit and reasonable bitrate when performing the encoding:
ffmpeg -stream_loop -1 -re -i 'AAC 5.1.mp4' -map 0:a:0 -c:a libopus -b:a 256k -f mpegts "srt://localhost:8890?streamid=publish:mystream"
Please test the latest feature/webrtc-multiopus
and let me know whether the streaming to the browser is successful. Thanks.
This is fantastic news, thank you so much. I tested, everything works perfectly. I run server from this branch and I can hear all the channels
Even 7.1 works perfectly
I tested locally and it worked perfectly on 5.1 and 7.1 (Downloaded from same source AAC 7.1.mp4
Google Drive folder)
Then I deployed it on external server with public IP.
5.1 worked as expected, but on 7.1 with WebRTC got deadline exceeded while waiting connection
, even thought I put webrtcAdditionalHosts
.
here is debug log (Masked IPs)
debug.txt
Publish command
ffmpeg -re -stream_loop 100 -i "/home/dito/Downloads/AAC 7.1.mp4" -c:v libx264 -bf 0 -c:a libopus -b:a 256k -f mpegts "srt://MEDIAMTX_PUBLIC_IP:8890?streamid=publish:mystream"
output
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: mp42isom
encoder : Lavf60.16.100
Stream #0:0(und): Audio: opus, 48000 Hz, 7.1, flt, 256 kb/s (default)
Metadata:
creation_time : 2013-05-09T11:31:26.000000Z
handler_name : soun
vendor_id : [0][0][0][0]
encoder : Lavc60.31.102 libopus
srt reciever
ffplay "srt://MEDIAMTX_PUBLIC_IP:8890?streamid=read:mystream"
output
Input #0, mpegts, from 'srt://MEDIAMTX_PUBLIC_IP:8890?streamid=read:mystream':
Duration: N/A, start: 58.325322, bitrate: N/A
Program 1
Stream #0:0[0x100]: Audio: opus (Opus / 0x7375704F), 48000 Hz, 7.1, fltp
works.
Then I downloaded dolby-demo-trailers-hd
And it worked with WebRTC as well
ffmpeg -re -stream_loop 100 -i "/home/dito/Downloads/7.1.mkv" -c:v libx264 -bf 0 -c:a libopus -b:a 256k -f mpegts "srt://MEDIAMTX_PUBLIC_IP:8890?streamid=publish:mystream"
output
Stream #0:0(eng): Video: h264, yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 29.97 fps, 90k tbn (default)
Metadata:
encoder : Lavc60.31.102 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Stream #0:1(eng): Audio: opus, 48000 Hz, 7.1, flt, 256 kb/s (default)
Metadata:
encoder : Lavc60.31.102 libopus
So it seams working Thank you very much
This issue is mentioned in release v1.8.3 🚀 Check out the entire changelog by clicking here
Describe the feature
Follows #3355