Closed limjoe closed 4 years ago
Looking forward to the author taking a look at this issue, thank you. This issue occurs when muting the device and then re-enabling the microphone, causing playback issues in Web player SDKs like AliPlayer.
TRANS_BY_GPT3
Attempt with only video scenario, conclusion when video changes:
Below are the steps:
ffmpeg -re -i doc/source.200kbps.768x320.flv -vcodec libx264 -s 512x400 -b:v 1500k -r 25 -profile:v baseline -an -f flv -y rtmp://localhost/live/livestream
ffmpeg -re -i doc/source.200kbps.768x320.flv -vcodec libx264 -s 304x208 -b:v 600k -r 20 -profile:v high -an -f flv -y rtmp://localhost/live/livestream
FFMPEG, VLC, and Flash can all successfully pull the video. This shows that there is no problem with the player restarting playback after changing the video.
Note that if VLC is not disconnected, changing the codec and video size during streaming will result in a distorted screen.
The first time VLC is like this:
After the second streaming, VLC (if not disconnected) is like this:
After disconnecting VLC and playing, it is normal.
It can be seen that if the playback is not disconnected, VLC will have screen flickering after changing the video, while Flash is normal.
TRANS_BY_GPT3
Trying with both audio and video, keeping the codec unchanged, changing from audio and video to pure video, the conclusions are as follows:
Below are the steps:
ffmpeg -re -i doc/source.200kbps.768x320.flv -vcodec copy -acodec copy -f flv -y rtmp://localhost/live/livestream
ffmpeg -re -i doc/source.200kbps.768x320.flv -vcodec copy -an -f flv -y rtmp://localhost/live/livestream
At this time, VLC playback fails while Flash playback succeeds.
Using srs_rtmp_dump to view server data:
[root@093b9be1d67d trunk]# ./objs/research/librtmp/srs_rtmp_dump -r rtmp://127.0.0.1:1935/live/livestream
[T][331][2020-03-12 03:13:29.963] Audio packet id=6/50.8/19.7, type=Audio, dts=0, pts=0, ndiff=2, diff=0, size=4, AAC(44KHz,16bit,Stereo,SH), (
0xaf 0x00 0x12 0x10 )
[T][331][2020-03-12 03:13:29.963] Video packet id=7/44.9/22.3, type=Video, dts=0, pts=0, ndiff=1, diff=0, size=46, H.264(SH,I), (0x17 0x00 0x00
0x00 0x00 0x01 0x64 0x00)
[T][331][2020-03-12 03:13:29.963] Video packet id=8/40.4/24.8, type=Video, dts=0, pts=80, ndiff=5, diff=0, size=5137, H.264(Nalu,I), (0x17 0x01
0x00 0x00 0x50 0x00 0x00 0x02)
It can be observed that the SequenceHeader of the audio is cached. At this point, since there is no audio available, it causes VLC to fail in synchronizing the audio and video.
If you wait long enough (around 10 seconds), the video can be played.
TRANS_BY_GPT3
Well, but in actual usage, the waiting time for opening the live broadcast is too long, which will result in a poor user experience or be considered a bug.
TRANS_BY_GPT3
In the case of pure audio, changing the encoding alters the behavior of the player. Conclusion:
The steps are as follows:
ffmpeg -re -i doc/source.200kbps.768x320.flv -vn -acodec libfdk_aac -ar 44100 -ac 2 -b:a 100k -f flv -y rtmp://localhost/live/livestream
ffmpeg -re -i doc/source.200kbps.768x320.flv -vn -acodec libfdk_aac -ar 32000 -ac 1 -b:a 64k -f flv -y rtmp://localhost/live/livestream
FFMPEG, VLC, and Flash can all successfully pull the stream. This indicates that there is no issue with the player restarting after changing the audio.
Note that if VLC is not disconnected, changing the codec during streaming can result in audio abnormalities.
The first time VLC is like this:
After the second streaming, VLC (if not disconnected) looks like this, but the audio is distorted:
After disconnecting VLC and playing, it is normal.
It can be seen that if the playback is not disconnected, VLC will have audio abnormalities after changing the audio encoding, while Flash is normal.
TRANS_BY_GPT3
In conclusion, the conclusion is as follows:
TRANS_BY_GPT3
SRS cache video/audio sequence header has a purpose to avoid sending duplicate SequenceHeader to the same client. For example, if the player remains unchanged but the encoder re-pushes with the same encoding parameters, some players may encounter issues if the encoding and decoding information is directly sent to the client.
// whether consumer should drop for the duplicated sequence header.
bool drop_for_reduce = false;
if (is_sequence_header && meta->vsh() && _srs_config->get_reduce_sequence_header(req->vhost)) {
if (meta->vsh()->size == msg->size) {
drop_for_reduce = srs_bytes_equals(meta->vsh()->payload, msg->payload, msg->size);
srs_warn("drop for reduce sh video, size=%d", msg->size);
}
}
We need to separate these two functions.
previous_vsh
and previous_ash
to determine if it is a duplicate SequenceHeader
and update the previous
information when unpublish
and update_vsh
and update_ash
are executed.vsh
and ash
each time publish
is executed, but do not clear the previous
information. This way, the new player will not cache the previous stream information. play {
reduce_sequence_header on;
}
After verification, this feature is also working fine.
TRANS_BY_GPT3
Do not change the encoding header, especially for VLC testing, as follows:
Note: Changing the encoding header will definitely cause problems with VLC. We will no longer test it as this is an issue with VLC.
Push audio and video streams, then only push the video stream, and then only push the audio stream to observe VLC behavior.
Push audio and video streams, VLC waits for about 3 seconds to play:
ffmpeg -ss 60s -re -i doc/source.200kbps.768x320.flv -vcodec copy -acodec copy -f flv -y rtmp://localhost/live/livestream
Only push the audio stream, VLC recovers after about 20 seconds:
ffmpeg -ss 60s -re -i doc/source.200kbps.768x320.flv -vn -acodec copy -f flv -y rtmp://localhost/live/livestream
Only push the video stream, VLC recovers after about 20 seconds:
ffmpeg -ss 60s -re -i doc/source.200kbps.768x320.flv -vcodec copy -an -f flv -y rtmp://localhost/live/livestream
Push audio and video streams, VLC recovers quickly:
ffmpeg -ss 60s -re -i doc/source.200kbps.768x320.flv -vcodec copy -acodec copy -f flv -y rtmp://localhost/live/livestream
Note: VLC takes a longer time to recover when disabling streams because it has a larger cache. For example, when switching from an audio and video stream to a pure video stream, there will still be a certain amount of cache time.
Push the audio stream, then push the audio-video stream, and observe VLC behavior:
ffmpeg -ss 60s -re -i doc/source.200kbps.768x320.flv -vn -acodec copy -f flv -y rtmp://localhost/live/livestream
ffmpeg -ss 60s -re -i doc/source.200kbps.768x320.flv -vcodec copy -acodec copy -f flv -y rtmp://localhost/live/livestream
Note: It can be seen that VLC relies on the initial encoding information from the first playback. If there is no video, even if there is video information later on, the video still cannot be seen.
Replaying allows you to see the audio and video streams.
Push video stream, then push audio and video stream, observe VLC behavior:
Only push video stream, VLC starts playing after about 10 seconds:
ffmpeg -ss 60s -re -i doc/source.200kbps.768x320.flv -vcodec copy -an -f flv -y rtmp://localhost/live/livestream
Push audio and video stream, VLC recovers quickly but without audio:
ffmpeg -ss 60s -re -i doc/source.200kbps.768x320.flv -vcodec copy -acodec copy -f flv -y rtmp://localhost/live/livestream
Note: Same as above, audio cannot be heard. VLC relies on the codec information obtained during the first time.
VLC has no problem with repeated Sequence Headers. It can handle repeated ones, but not different ones.
Note: Flash also has no problem with repeated Sequence Headers.
TRANS_BY_GPT3
In summary: VLC relies on the initial encoding header information. Although SRS provides the correct encoding header, it still cannot correctly recognize the newly added stream.
TRANS_BY_GPT3
Description'
Please ensure that you maintain the markdown structure.
Please ensure that you maintain the markdown structure.
3.0.121
Please ensure that you maintain the markdown structure.
Please ensure that you maintain the markdown structure.
Replay
Please ensure that you maintain the markdown structure.
How to replay bug? The pure video data file used for replay can be found at: link (password: cnvp). Please ensure that you maintain the markdown structure.
1. Pushing pure video stream data to channel A using ffmpeg, the rtmp playback is normal, and VLC's meta information also shows that only the video type stream is present. Please ensure that you maintain the markdown structure.
ffmpeg -re -i 1583334009334.flv -vcodec copy -f flv -y rtmp://localhost:1935/live/LIVEPL16DD80CE047_2
1. Whether using ffmpeg or pushing data containing both audio and video to ChannelA, the rtmp playback is normal, and VLC's meta information will display two streams, one for audio and one for video. Please ensure that you maintain the markdown structure.ffmpeg -re -i 1583334009334.flv -vcodec copy -acodec copy -f flv -y rtmp://localhost:1935/live/LIVEPL16DD80CE047_2
**1. Repeating the operation in step 1, pushing a stream of pure video data to Channel A results in abnormal rtmp playback (VLC cannot play, it remains black after a long buffering time). In VLC's meta information, besides displaying the video's meta information, an additional audio stream meta information will also be displayed.Expected behavior
Expected behavior:
TRANS_BY_GPT3