search

harlanc / xiu

A simple,high performance and secure live media server in pure Rust (RTMP[cluster]/RTSP/WebRTC[whip/whep]/HTTP-FLV/HLS).🦀
https://rustxiu.com
MIT License
1.76k stars 187 forks source link

The video and audio are out of sync #58

Closed GitNiko closed 10 months ago

GitNiko commented 1 year ago

At first, thanks for your awesome project. I've encountered an issue where I've noticed that the video and audio are not synchronized when playing. Has anyone else experienced this problem? I confirm that my source video is right. My enviroment:

Logs:

[2023-09-25T09:30:23Z INFO  httpflv::server] Httpflv server listening on http://0.0.0.0:8090
[2023-09-25T09:30:23Z INFO  hls::server] Hls server listening on http://0.0.0.0:8091
[2023-09-25T09:30:23Z INFO  xrtsp::rtsp] Rtsp server listening on tcp://0.0.0.0:5544
[2023-09-25T09:30:23Z INFO  rtmp::remuxer] rtmp remuxer start...
[2023-09-25T09:30:23Z INFO  rtmp::rtmp] Rtmp server listening on tcp://0.0.0.0:1935
[2023-09-25T09:30:23Z INFO  xiu::api] Http api server listening on http://:8000

[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] server session: 172.19.132.133:61383
[2023-09-25T09:30:35Z INFO  rtmp::handshake::handshake_server] [ S<-C ] [complex handshake] read C0C1
[2023-09-25T09:30:35Z INFO  rtmp::handshake::handshake_server] [ S->C ] [complex handshake] write S0S1S2
[2023-09-25T09:30:35Z INFO  rtmp::handshake::handshake_server] [ S->C ] [complex handshake] write S0S1S2 finish
[2023-09-25T09:30:35Z INFO  rtmp::handshake::handshake_server] [ S<-C ] [complex handshake] read C2
[2023-09-25T09:30:35Z INFO  rtmp::handshake::handshake_server] complex handshake successfully..
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S->C ] [send_set_chunk_size] 
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S<-C ] [connect] 
[2023-09-25T09:30:35Z WARN  rtmp::session::server_session] unknown connect properties: type:UTF8String("nonprivate")
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] connect properties: ConnectProperties { app: Some("live"), flash_ver: Some("FMLE/3.0 (compatible; Lavf59.27.100)"), swf_url: None, tc_url: Some("rtmp://10.20.0.18:1935/live"), fpad: None, capabilities: None, audio_codecs: None, video_codecs: None, video_function: None, object_encoding: None, page_url: None, pub_type: None }
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S->C ] [set window_acknowledgement_size]
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S->C ] [set set_peer_bandwidth]
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S->C ] [set connect_response]
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S<-C ] [set chunk size]  app_name: live, stream_name: , chunk size: 4096
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S<-C ] [create stream] 
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S->C ] [create_stream_response]  app_name: live
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S<-C ] [publish]  app_name: live, stream_name: test, url parameters: 
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S->C ] [stream begin]  app_name: live, stream_name: test, url parameters: 
[2023-09-25T09:30:35Z INFO  rtmp::session::server_session] [ S->C ] [NetStream.Publish.Start]  app_name: live, stream_name: test
[2023-09-25T09:30:35Z INFO  streamhub] event data: {"Publish":{"identifier":{"Rtmp":{"app_name":"live","stream_name":"test"}},"info":{"id":"16956342354706","sub_type":"PushRtmp","notify_info":{"request_url":"rtmp://10.20.0.18:1935/live/test","remote_addr":"172.19.132.133:61383"}}}}
[2023-09-25T09:30:35Z INFO  rtmp::remuxer] Publish { identifier: Rtmp { app_name: "live", stream_name: "test" } }
[2023-09-25T09:30:35Z INFO  streamhub] event data: {"Subscribe":{"identifier":{"Rtmp":{"app_name":"live","stream_name":"test"}},"info":{"id":"16956342355981","sub_type":"GenerateHls","notify_info":{"request_url":"","remote_addr":""}}}}
[2023-09-25T09:30:35Z INFO  streamhub] subscribe:  stream identifier: RTMP - app_name: live, stream_name: test
[2023-09-25T09:30:35Z INFO  rtmp::cache::metadata] metadata: [UTF8String("@setDataFrame"), UTF8String("onMetaData"), Object({"width": Number(720.0), "height": Number(1280.0), "videodatarate": Number(1335.7568359375), "framerate": Number(25.0), "videocodecid": Number(7.0), "audiodatarate": Number(63.845703125), "audiosamplerate": Number(24000.0), "audiosamplesize": Number(16.0), "stereo": Boolean(false), "audiocodecid": Number(10.0), "major_brand": UTF8String("isom"), "minor_version": UTF8String("512"), "compatible_brands": UTF8String("isomiso2avc1mp41"), "comment": UTF8String("{\"top_left\": [175, 163], \"size\": 430, \"detection\": [285, 271, 497, 545], \"face_id\": 0, \"detect_confidence\": 0.9998562335968018, \"mask_confidence\": 1.0, \"overlap\": \"no\", \"neighbors\": null}"), "encoder": UTF8String("Lavf59.27.100")})]
[2023-09-25T09:30:35Z INFO  h264_decoder::sps] profile_idc: 77
[2023-09-25T09:30:35Z INFO  h264_decoder::sps] level_idc: 31
[2023-09-25T09:30:35Z INFO  xflv::mpeg4_avc] mpeg4 avc profile: 77
[2023-09-25T09:30:35Z INFO  xflv::mpeg4_avc] mpeg4 avc compatibility: 64
[2023-09-25T09:30:35Z INFO  xflv::mpeg4_avc] mpeg4 avc level: 31
[2023-09-25T09:30:35Z INFO  xflv::mpeg4_avc] mpeg4 avc resolution: 720x1280
[2023-09-25T09:30:35Z INFO  h264_decoder::sps] profile_idc: 77
[2023-09-25T09:30:35Z INFO  h264_decoder::sps] level_idc: 31
[2023-09-25T09:30:35Z INFO  xflv::mpeg4_avc] mpeg4 avc profile: 77
[2023-09-25T09:30:35Z INFO  xflv::mpeg4_avc] mpeg4 avc compatibility: 64
[2023-09-25T09:30:35Z INFO  xflv::mpeg4_avc] mpeg4 avc level: 31
[2023-09-25T09:30:35Z INFO  xflv::mpeg4_avc] mpeg4 avc resolution: 720x1280
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] server session: 172.19.132.133:61391
[2023-09-25T09:31:11Z INFO  rtmp::handshake::handshake_server] [ S<-C ] [complex handshake] read C0C1
[2023-09-25T09:31:11Z INFO  rtmp::handshake::handshake_server] [ S->C ] [complex handshake] write S0S1S2
[2023-09-25T09:31:11Z INFO  rtmp::handshake::handshake_server] [ S->C ] [complex handshake] write S0S1S2 finish
[2023-09-25T09:31:11Z INFO  rtmp::handshake::handshake_server] [ S<-C ] [complex handshake] read C2
[2023-09-25T09:31:11Z INFO  rtmp::handshake::handshake_server] complex handshake successfully..
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S->C ] [send_set_chunk_size] 
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S<-C ] [connect] 
[2023-09-25T09:31:11Z WARN  rtmp::session::server_session] unknown connect properties: capabilities:Number(15.0)
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] connect properties: ConnectProperties { app: Some("live"), flash_ver: Some("LNX 9,0,124,2"), swf_url: None, tc_url: Some("rtmp://10.20.0.18:1935/live"), fpad: Some(false), capabilities: None, audio_codecs: Some(4071.0), video_codecs: Some(252.0), video_function: Some(1.0), object_encoding: None, page_url: None, pub_type: None }
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S->C ] [set window_acknowledgement_size]
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S->C ] [set set_peer_bandwidth]
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S->C ] [set connect_response]
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S<-C ] [create stream] 
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S->C ] [create_stream_response]  app_name: live
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S<-C ] [play]  app_name: live, stream_name: 
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S->C ] [stream begin]  app_name: live, stream_name: 
[2023-09-25T09:31:11Z INFO  rtmp::session::server_session] [ S->C ] [stream is record]  app_name: live, stream_name: test, url parameters: 
[2023-09-25T09:31:11Z INFO  rtmp::session::common] subscribe_from_channels, app_name: live stream_name: test subscribe_id: 16956342712350
[2023-09-25T09:31:11Z INFO  streamhub] event data: {"Subscribe":{"identifier":{"Rtmp":{"app_name":"live","stream_name":"test"}},"info":{"id":"16956342712350","sub_type":"PlayerRtmp","notify_info":{"request_url":"rtmp://10.20.0.18:1935/live/test","remote_addr":"172.19.132.133:61391"}}}}
[2023-09-25T09:31:11Z INFO  streamhub] subscribe:  stream identifier: RTMP - app_name: live, stream_name: test
harlanc commented 1 year ago

Thanks for your report, could you please upload the mp4 file? And I can do some check.

GitNiko commented 1 year ago

Hi, You can check this video

After about 5 or 6 rounds of looping, You can notice an issue with audio and video synchronization. I didn't encounter this problem when I used OBS for live streaming. So, I strongly suspect it might be an issue related to ffmpeg -re -stream_loop -1 -i talking.mp4 -c:a copy -c:v copy -f flv -flvflags no_duration_filesize rtmp://10.20.0.18:1935/live/test.

harlanc commented 1 year ago

It looks like a bug, needs to be fixed.

harlanc commented 10 months ago

It seems fixed in branch fix_71.