Audio Video Out-of-sync when using ffmpeg+shaka-packager

[sridhard]

System info

Operating System: <e.g. macOS Sierra, Ubuntu 14.04 trusty etc> Ubuntu 18

Shaka Packager Version: <e.g. v1.6.1, commit SHA etc> 2.4.1

Issue and steps to reproduce the problem

I am using ffmpeg to receive rtmp stream. ffmpeg output is passed to shaka-packager via udp. shaka-packager will convert to DASH and the DASH stream is played using shaka-player. Always I find that in shaka-player the audio-video is out-of-sync.

Below are the commands: ffmpeg -y -listen 1 -thread_queue_size 200 -i "rtmp://127.0.0.1:1437/live/abc" -an -preset veryfast -pix_fmt yuv420p -flags +cgop -c:v h264 -profile:v baseline -level 3.0 -r 30 -crf 23 -keyint_min 30 -g 30 -vf scale=-2:640,setsar=1:1 -f mpegts "udp://127.0.0.1:40000" -an -preset veryfast -pix_fmt yuv420p -flags +cgop -c:v h264 -profile:v baseline -level 3.0 -r 30 -crf 23 -keyint_min 30 -g 30 -vf scale=-2:360,setsar=1:1 -f mpegts "udp://127.0.0.1:40001"

packager "in=udp://127.0.0.1:40000,stream=video,init_segment=./livedata/HQVideo_init.mp4,segmenttemplate=./livedata/HQVideo$Number$.m4s" --segment_duration 1 "in=udp://127.0.0.1:40001,stream=video,init_segment=./livedata/MQVideo_init.mp4,segmenttemplate=./livedata/MQVideo$Number$.m4s" --segment_duration 1 --mpd_output "./livedata/$1/$2/$3/livestream1.mpd" --allow_approximate_segment_timeline --segment_template_constant_duration

To verify whether the problem is with ffmpeg or with input stream I have verified the below:

1. Receive RTMP in ffmpeg and directly write to mp4 file. Here there is no audio-video sync issue
2. Receive RTMP in ffmoeg and write audio and video as mpegts files seperately. Afte live merge the files and check. There is no audio-video sync issue
3. Receive RTMP in ffmpeg and ffm[eg write the dash stream directly. Play the stream live in shaka-player. There is no audio-video sync issue

But if we send the stream to shaka-packager, shaka-packager convert to DASH and play DASH stream live then always there is audio-video sync issue.

In shaka-packager always I get the below error logs: stderr] [0406/100937:WARNING:es_parser_h26x.cc(334)] [MPEG-2 TS] PID 256 Possible GAP at dts 126000 with next sample at dts 129750 (difference 3750) [stderr] [0406/100939:WARNING:es_parser_h26x.cc(334)] [MPEG-2 TS] PID 256 Possible GAP at dts 126000 with next sample at dts 129000 (difference 3000)

Also, In shaka player always the LIVE stream start time is 1.4 seconds. What I think is there is a loss of data for 1.4 sec which is causing the sync issue.

Can you please let me know how to debug this issue? Also what data I need to attach in this bug so that it will be helpful for debugging.

Packager Command: Given above

Extra steps to reproduce the problem? NA

What is the expected result? There should not be any audio-video sync issue

What happens instead? audio-video sync issue in packager putput

<Please attach the input files or email to shaka-packager-issues@google.com.>

[kqyang]

@sridhard A few questions:

• Are you seeing a/v out of sync since the beginning? Or does it only happen after running for a while?
• How frequently are you seeing the "gap" warning?
• Can you estimate the timing difference between a/v? 1.4 seconds?

It is a bit hard to debug with rtmp stream. What happens if you use ffmpeg to write rtmp to mp4 or ts file and then feed the mp4 or ts file to Shaka Packager? Will you still be seeing a/v out of sync in this case?

[sridhard]

@kqyang thanks. Sorry for late reply. Below are the answers

Audio-video are in sync in the beginning and it happens over a period of time. Gaps are reported by shaka packager approximately every 4-5 seconds. Please check the packager logs https://share-to-vijay.s3.ap-south-1.amazonaws.com/shakapackagertests/packager-errors.txt

The timing difference is not 1.4 sec and it happens over a period of time. May be the Gaps are causing the audio sync issue? how can we solve these Gaps?

[kqyang]

May be the Gaps are causing the audio sync issue?

Possible. I am seeing about one video frame loss only though in your log. Is the A/V out of sync only by one video frame (0.033 seconds)?

how can we solve these Gaps?

We'll need to understand what is causing the gap.

What happens if you use ffmpeg to write rtmp to mp4 or ts file and then feed the mp4 or ts file to Shaka Packager? Will you still be seeing a/v out of sync in this case?

Have you tried the above?

[sridhard]

@kqyang rtmp->mp4 and then play mp4 it is ok. No issues. When we pacjage mp4 with shaka-packager also no issues.

[kqyang]

How about rtmp->ts and then package the ts with shaka-packager?

[Nixon197]

@sridhard you should try ffmpeg piping: https://google.github.io/shaka-packager/html/tutorials/ffmpeg_piping.html

You could be experiencing udp packet loss.

[aleek]

Try playling with ffmpeg's fps filter, i.e. fps=fps=30:start_time=0:round=near. AFAIR, it helped me with similar problem.

[alexdjca]

I'm having exactly the same problem and unfortunately i cannot even try to pipe because my transcoders and packagers are in different machines, or to set a static FPS because in transcoding i have to keep the original frame rate received in input. Even if i was enthusiastic to use shaka-packager i will probably need to use ffmpeg also for packaging :(

[alexdjca]

forgot to mention that my chain is: mpegts ---> ffmpeg ----> 4x mpegts ----> shaka-packager

my ffmpeg encoding line is:

/usr/bin/ffmpeg -hwaccel nvdec -i udp://239.3.100.54:8000?fifo_size=1000000 -filter_complex "[0:v]yadif, split=5[SDv][SDhv][SDqv][HDv][FHDv];[SDv]scale=-1:576[SDvINT];[SDhv]scale=-1:480[SDhvINT];[SDqv]scale=-1:360[SDqvINT];[HDv]scale=-1:720[HDvINT];[FHDv]scale=-1:1080[FHDvINT]" -async 1 -vsync 1 -g 60 -c:v h264_nvenc -c:a aac -map [SDvINT] -map [SDhvINT] -map [SDqvINT] -map [HDvINT] -map [FHDvINT] -b:v:0 1400k -b:v:1 700k -b:v:2 300k -b:v:3 2500k -profile:v:3 high -level:v:3 4.1 -b:v:4 3200k -profile:v:4 high -level:v:4 4.1 -map 0:a -map 0:a -b:a:0 128k -b:a:1 64k -f tee [select=\'v:0,a:0,d:0\':f=mpegts]udp://239.5.208.109:4000?pkt_size=1316|[select=\'v:1,a:1,d:0\':f=mpegts]udp://239.5.208.109:3000?pkt_size=1316|[select=\'v:2,a:1,d:0\':f=mpegts]udp://239.5.208.109:2000?pkt_size=1316|[select=\'v:3,a:0,d:0\':f=mpegts]udp://239.5.208.109:5000?pkt_size=1316|[select=\'v:4,a:0,d:0\':f=mpegts]udp://239.5.208.109:6000?pkt_size=1316

i've also tried without -async 1 -vsync 1 or with -async 1 -vsync 0 and -async 0 -vsync 1

but whatever i do, the audio in the HLS output of shaka-packager is always out of sync.

The ouput in the mpegts output of FFMPEG plays perfectly fine

[Nixon197]

Hi

Can you try adding this audio filter to your ffmpeg comand -af "aresample=async=1:min_hard_comp=0.100000:first_pts=0"

Br

Niksa

[Brainiarc7]

Hello there,

Try this command instead:

/usr/bin/ffmpeg -threads 1 -i udp://239.3.100.54:8000?fifo_size=1000000 -filter_complex "[0:v]realtime,hwupload_cuda=0,yadif_cuda=0:-1:0, split=5[SDv][SDhv][SDqv][HDv][FHDv];[SDv]scale_cuda=-1:576[SDvINT];[SDhv]scale_cuda=-1:480[SDhvINT];[SDqv]scale_cuda=-1:360[SDqvINT];[HDv]scale_cuda=-1:720[HDvINT];[FHDv]scale_cuda=-1:1080[FHDvINT]" -async 1 -vsync 1 -g 60 -c:v h264_nvenc -gpu 0 -c:a aac -map [SDvINT] -map [SDhvINT] -map [SDqvINT] -map [HDvINT] -map [FHDvINT] -b:v:0 1400k -b:v:1 700k -b:v:2 300k -b:v:3 2500k -profile:v:3 high -level:v:3 4.1 -b:v:4 3200k -profile:v:4 high -level:v:4 4.1 -map 0:a -map 0:a -b:a:0 128k -b:a:1 64k -f tee [select=\'v:0,a:0,d:0\':f=mpegts]udp://239.5.208.109:4000?pkt_size=1316|[select=\'v:1,a:1,d:0\':f=mpegts]udp://239.5.208.109:3000?pkt_size=1316|[select=\'v:2,a:1,d:0\':f=mpegts]udp://239.5.208.109:2000?pkt_size=1316|[select=\'v:3,a:0,d:0\':f=mpegts]udp://239.5.208.109:5000?pkt_size=1316|[select=\'v:4,a:0,d:0\':f=mpegts]udp://239.5.208.109:6000?pkt_size=1316

Changes I made:

1. I disabled NVDEC. That code path will only give you headaches down the path with UDP input. Secondly, its' not necessarily faster than software-based decode. Note that in ffmpeg, software-based decoding (typically) scales very poorly with multiple threads for codecs such as H.264/AVC. Stick to one thread only (see -threads 1 option above).

2. See the use of hwupload_cuda. That filter creates a CUDA device from which scale_cuda can operate from transparently with zero performance overhead, unlike the generic hwupload filter. I picked the scale_cuda filter instead of scale_npp because without your build config (see ffmpeg -buildconf), I cannot assume that you have CUDA SDK options enabled. However, with NVENC enabled,scale_cuda will also be available as it depends on the ffnvcodec headers which you already have. The other filter scale_npp needs the proprietary CUDA SDK to be installed.

3. I also passed the private codec option -gpu 0 to the nvenc encoder. That way, only GPU 0 will be used for transcoding, matching up to the same device index as the CUDA device we created for the filters. No unneccessary texture download overheads are incurred.

4. A realtime filter was also added to the complex filter chain. Note its' position. For more on what's available with NVIDIA-specific hardware-acceleration in FFmpeg (filters, decoding, encoding and related wrappers), see my answer here.

Results:

Your output should now be realtime (1x) at all times, and with lower CPU overhead too, and this should also fix your audio sync issues.

Test and report back please.

[alexdjca]

Hi Niksa, unfortunately your suggestion did produce any difference :(

Hi Dennis,

My ffmpeg doesn't support the yadif_cuda filter, neither scale_cuda

ffmpeg -filters | grep scale ... alphaextract V->N Extract an alpha channel as a grayscale image component. ... extractplanes V->N Extract planes as grayscale frames. ..C scale V->V Scale the input video size and/or convert the image format. ... scale_vaapi V->V Scale to/from VAAPI surfaces. ..C scale2ref VV->VV Scale the input video size and/or convert the

ffmpeg -filters | grep yadif TS. yadif V->V Deinterlace the input image.

And i cannot even compile ffmpeg because this is just one of multiple encoders in which several people are involved in the management and in this case we need to stick with what's provided with the distro used (Ubuntu 20.04LTS).

If i could i would definitely use the proprietary NPP/CUDA_SDK rather than the current settings, but by experience this will cause a lot of headache in the day to day operations (eg. kernel updates and so on)... It already happened.

Anyway, i'm going ahead to test your suggestion on another server. But bear in mind that the UDP outputs of the encoder are already perfect, the desync for some reason happens when i convert in HLS with the shaka-packager.

I will keep you posted in any case.

Thanks Alex

On Sat, 27 Jun 2020 at 00:19, Dennis E. Mungai notifications@github.com wrote:

Hello there,

Try this command instead:

/usr/bin/ffmpeg -threads 1 -i udp://239.3.100.54:8000?fifo_size=1000000 -filter_complex "[0:v]realtime,hwupload_cuda=0,yadif_cuda=0:-1:0, split=5[SDv][SDhv][SDqv][HDv][FHDv];[SDv]scale_cuda=-1:576[SDvINT];[SDhv]scale_cuda=-1:480[SDhvINT];[SDqv]scale_cuda=-1:360[SDqvINT];[HDv]scale_cuda=-1:720[HDvINT];[FHDv]scale_cuda=-1:1080[FHDvINT]" -async 1 -vsync 1 -g 60 -c:v h264_nvenc -gpu 0 -c:a aac -map [SDvINT] -map [SDhvINT] -map [SDqvINT] -map [HDvINT] -map [FHDvINT] -b:v:0 1400k -b:v:1 700k -b:v:2 300k -b:v:3 2500k -profile:v:3 high -level:v:3 4.1 -b:v:4 3200k -profile:v:4 high -level:v:4 4.1 -map 0:a -map 0:a -b:a:0 128k -b:a:1 64k -f tee [select=\'v:0,a:0,d:0\':f=mpegts]udp://239.5.208.109:4000?pkt_size=1316|[select=\'v:1,a:1,d:0\':f=mpegts]udp://239.5.208.109:3000?pkt_size=1316|[select=\'v:2,a:1,d:0\':f=mpegts]udp://239.5.208.109:2000?pkt_size=1316|[select=\'v:3,a:0,d:0\':f=mpegts]udp://239.5.208.109:5000?pkt_size=1316|[select=\'v:4,a:0,d:0\':f=mpegts]udp://239.5.208.109:6000?pkt_size=1316

Changes I made:

1.

I disabled NVDEC. That code path will only give you headaches down the path with UDP input. Secondly, its' not necessarily faster than software-based decode. Note that in ffmpeg, software-based decoding (typically) scales very poorly with multiple threads for codecs such as H.264/AVC. Stick to one thread only (see -threads 1 option above). 2.

See the use of hwupload_cuda. That filter creates a CUDA device from which scale_cuda can operate from transparently with zero performance overhead, unlike the generic hwupload filter. I picked the scale_cuda filter instead of scale_npp because without your build config (see ffmpeg -buildconf), I cannot assume that you have CUDA SDK options enabled. However, with NVENC enabled, scale_cuda will also be available as it depends on the ffnvcodec https://github.com/FFmpeg/nv-codec-headers/tree/master/include/ffnvcodec headers which you already have. The other filter scale_npp needs the proprietary CUDA SDK to be installed. 3.

I also passed the private codec option -gpu 0 to the nvenc encoder. That way, only GPU 0 will be used for transcoding, matching up to the same device index as the CUDA device we created for the filters. No unneccessary texture download overheads are incurred. 4.

A realtime filter was also added to the complex filter chain. Note its' position. For more on what's available with NVIDIA-specific hardware-acceleration in FFmpeg (filters, decoding, encoding and related wrappers), see my answer here https://stackoverflow.com/questions/55687189/how-to-use-gpu-to-accelerate-the-processing-speed-of-ffmpeg-filter/55747785#55747785 .

Results:

Your output should now be realtime (1x) at all times, and with lower CPU overhead too, and this should also fix your audio sync issues.

Test and report back please.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/shaka-packager/issues/746#issuecomment-650446214, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL73MKR3I3MVV3WIH7W4ALRYUUHVANCNFSM4MCERZFQ .

[alexdjca]

Hi Again,

I've tested this two commands, but the result is exactly the same as before. Audio and Video synced in the UDP output, but desynced after Skaka-packager.

test 1:

./ffmpeg -threads 1 -i udp://239.3.100.54:8000?fifo_size=1000000 -filter_complex "[0:v]realtime,hwupload_cuda=0,yadif_cuda=0:-1:0,split=5[SDv][SDhv][SDqv][HDv][FHDv];[SDv]scale_cuda=-1:576[SDvINT];[SDhv]scale_cuda=-1:480[SDhvINT];[SDqv]scale_cuda=-1:360[SDqvINT];[HDv]scale_cuda=-1:720[HDvINT];[FHDv]scale_cuda=-1:1080[FHDvINT]" -async 1 -vsync 1 -g 60 -c:v h264_nvenc -gpu 0 -c:a aac -map [SDvINT] -map
[SDhvINT] -map [SDqvINT] -map [HDvINT] -map [FHDvINT] -b:v:0 1400k -b:v:1 700k -b:v:2 300k -b:v:3 2500k -profile:v:3 high -level:v:3 4.1 -b:v:4 3200k -profile:v:4 high -level:v:4 4.1 -map 0:a -map 0:a -b:a:0 128k -b:a:1 64k
-f tee "[select=\'v:0,a:0,d:0\':f=mpegts]udp://239.5.208.109:4000?pkt_size=1316|[select=\'v:1,a:1,d:0\':f=mpegts]udp://239.5.208.109:3000?pkt_size=1316|[select=\'v:2,a:1,d:0\':f=mpegts]udp://239.5.208.109:2000?pkt_size=1316|[select=\'v:3,a:0,d:0\':f=mpegts]udp://239.5.208.109:5000?pkt_size=1316|[select=\'v:4,a:0,d:0\':f=mpegts]udp://239.5.208.109:6000?pkt_size=1316"

test 2:

./ffmpeg -threads 1 -hwaccel cuvid -c:v h264_cuvid -i udp://239.3.100.54:8000?fifo_size=1000000 -filter_complex
"[0:v]realtime,split=5[SDv][SDhv][SDqv][HDv][FHDv];[SDv]scale_npp=-1:576[SDvINT];[SDhv]scale_npp=-1:480[SDhvINT];[SDqv]scale_npp=-1:360[SDqvINT];[HDv]scale_npp=-1:720[HDvINT];[FHDv]scale_npp=-1:1080[FHDvINT]" -async 1 -vsync 1 -g 60 -c:v h264_nvenc -gpu 0 -c:a aac -map [SDvINT] -map [SDhvINT] -map [SDqvINT] -map [HDvINT] -map [FHDvINT] -b:v:0 1400k -b:v:1 700k -b:v:2 300k -b:v:3 2500k -profile:v:3 high -level:v:3 4.1 -b:v:4 3200k -profile:v:4 high -level:v:4 4.1 -map 0:a -map 0:a -b:a:0 128k -b:a:1 64k -f tee "[select=\'v:0,a:0,d:0\':f=mpegts]udp://239.5.208.109:4000?pkt_size=1316|[select=\'v:1,a:1,d:0\':f=mpegts]udp://239.5.208.109:3000?pkt_size=1316|[select=\'v:2,a:1,d:0\':f=mpegts]udp://239.5.208.109:2000?pkt_size=1316|[select=\'v:3,a:0,d:0\':f=mpegts]udp://239.5.208.109:5000?pkt_size=1316|[select=\'v:4,a:0,d:0\':f=mpegts]udp://239.5.208.109:6000?pkt_size=1316
"

Please note that since i didn't have yadif_npp compiled i even tried directly removing the deinterlace completely.

The FFMPEG version i've used now is the one i'm using in my personal lab:

ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 8 (Ubuntu 8.3.0-6ubuntu1)
  configuration: --enable-libnpp --enable-cuda --enable-gpl
--enable-version3 --enable-nonfree --enable-avresample --enable-avisynth
--enable-gnutls --enable-libfontconfig --enable-libfreetype
--enable-libfribidi --enable-libmp3lame --enable-libopenjpeg
--enable-libopenmpt --enable-libshine --enable-libssh --enable-libvorbis
--enable-libvpx --enable-libx265 --enable-libx264 --enable-libfdk_aac
--enable-libxml2 --enable-libzvbi --enable-opengl --enable-libdc1394
--enable-libdrm --enable-chromaprint --enable-frei0r --enable-libx264
--enable-nvenc --enable-static --enable-libsrt --enable-cuda-nvcc

On Mon, 29 Jun 2020 at 17:11, Alex Molon molon.alex@gmail.com wrote:

Hi Niksa, unfortunately your suggestion did produce any difference :(

Hi Dennis,

My ffmpeg doesn't support the yadif_cuda filter, neither scale_cuda

ffmpeg -filters | grep scale ... alphaextract V->N Extract an alpha channel as a grayscale image component. ... extractplanes V->N Extract planes as grayscale frames. ..C scale V->V Scale the input video size and/or convert the image format. ... scale_vaapi V->V Scale to/from VAAPI surfaces. ..C scale2ref VV->VV Scale the input video size and/or convert the

ffmpeg -filters | grep yadif TS. yadif V->V Deinterlace the input image.

And i cannot even compile ffmpeg because this is just one of multiple encoders in which several people are involved in the management and in this case we need to stick with what's provided with the distro used (Ubuntu 20.04LTS).

If i could i would definitely use the proprietary NPP/CUDA_SDK rather than the current settings, but by experience this will cause a lot of headache in the day to day operations (eg. kernel updates and so on)... It already happened.

Anyway, i'm going ahead to test your suggestion on another server. But bear in mind that the UDP outputs of the encoder are already perfect, the desync for some reason happens when i convert in HLS with the shaka-packager.

I will keep you posted in any case.

Thanks Alex

On Sat, 27 Jun 2020 at 00:19, Dennis E. Mungai notifications@github.com wrote:

Hello there,

Try this command instead:

/usr/bin/ffmpeg -threads 1 -i udp://239.3.100.54:8000?fifo_size=1000000 -filter_complex "[0:v]realtime,hwupload_cuda=0,yadif_cuda=0:-1:0, split=5[SDv][SDhv][SDqv][HDv][FHDv];[SDv]scale_cuda=-1:576[SDvINT];[SDhv]scale_cuda=-1:480[SDhvINT];[SDqv]scale_cuda=-1:360[SDqvINT];[HDv]scale_cuda=-1:720[HDvINT];[FHDv]scale_cuda=-1:1080[FHDvINT]" -async 1 -vsync 1 -g 60 -c:v h264_nvenc -gpu 0 -c:a aac -map [SDvINT] -map [SDhvINT] -map [SDqvINT] -map [HDvINT] -map [FHDvINT] -b:v:0 1400k -b:v:1 700k -b:v:2 300k -b:v:3 2500k -profile:v:3 high -level:v:3 4.1 -b:v:4 3200k -profile:v:4 high -level:v:4 4.1 -map 0:a -map 0:a -b:a:0 128k -b:a:1 64k -f tee [select=\'v:0,a:0,d:0\':f=mpegts]udp://239.5.208.109:4000?pkt_size=1316|[select=\'v:1,a:1,d:0\':f=mpegts]udp://239.5.208.109:3000?pkt_size=1316|[select=\'v:2,a:1,d:0\':f=mpegts]udp://239.5.208.109:2000?pkt_size=1316|[select=\'v:3,a:0,d:0\':f=mpegts]udp://239.5.208.109:5000?pkt_size=1316|[select=\'v:4,a:0,d:0\':f=mpegts]udp://239.5.208.109:6000?pkt_size=1316

Changes I made:

1.

I disabled NVDEC. That code path will only give you headaches down the path with UDP input. Secondly, its' not necessarily faster than software-based decode. Note that in ffmpeg, software-based decoding (typically) scales very poorly with multiple threads for codecs such as H.264/AVC. Stick to one thread only (see -threads 1 option above). 2.

See the use of hwupload_cuda. That filter creates a CUDA device from which scale_cuda can operate from transparently with zero performance overhead, unlike the generic hwupload filter. I picked the scale_cuda filter instead of scale_npp because without your build config (see ffmpeg -buildconf), I cannot assume that you have CUDA SDK options enabled. However, with NVENC enabled, scale_cuda will also be available as it depends on the ffnvcodec https://github.com/FFmpeg/nv-codec-headers/tree/master/include/ffnvcodec headers which you already have. The other filter scale_npp needs the proprietary CUDA SDK to be installed. 3.

I also passed the private codec option -gpu 0 to the nvenc encoder. That way, only GPU 0 will be used for transcoding, matching up to the same device index as the CUDA device we created for the filters. No unneccessary texture download overheads are incurred. 4.

A realtime filter was also added to the complex filter chain. Note its' position. For more on what's available with NVIDIA-specific hardware-acceleration in FFmpeg (filters, decoding, encoding and related wrappers), see my answer here https://stackoverflow.com/questions/55687189/how-to-use-gpu-to-accelerate-the-processing-speed-of-ffmpeg-filter/55747785#55747785 .

Results:

Your output should now be realtime (1x) at all times, and with lower CPU overhead too, and this should also fix your audio sync issues.

Test and report back please.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/shaka-packager/issues/746#issuecomment-650446214, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABL73MKR3I3MVV3WIH7W4ALRYUUHVANCNFSM4MCERZFQ .

[Brainiarc7]

Try playling with ffmpeg's fps filter, i.e. fps=fps=30:start_time=0:round=near. AFAIR, it helped me with similar problem.

I'll test this and report back.

[Brainiarc7]

Hello there,

Try this:

/usr/bin/ffmpeg -vsync passthrough -async 1 -threads 1 \
-i "udp://239.3.100.54:8000?fifo_size=1000000" \
-filter_complex \
"[0:v]hwupload_cuda=0,yadif_cuda=0:-1:0,fps=fps=30:start_time=0:round=near,split=5[SDv][SDhv][SDqv][HDv][FHDv];[SDv]scale_cuda=-1:576[SDvINT];[SDhv]scale_cuda=-1:480[SDhvINT];[SDqv]scale_cuda=-1:360[SDqvINT];[HDv]scale_cuda=-1:720[HDvINT];[FHDv]scale_cuda=-1:1080[FHDvINT]" \
-g 60 -c:v h264_nvenc -gpu 0 -c:a aac -map [SDvINT] -map [SDhvINT] -map [SDqvINT] -map [HDvINT] -map [FHDvINT] \
-b:v:0 1400k -b:v:1 700k -b:v:2 300k -b:v:3 2500k -profile:v:3 high -level:v:3 4.1 -b:v:4 3200k -profile:v:4 high -level:v:4 4.1 \
-map 0:a -map 0:a -b:a:0 128k -b:a:1 64k \
-avoid_negative_ts disabled -muxdelay 0 \
-f tee "[select=\'v:0,a:0,d:0\':f=mpegts]udp://239.5.208.109:4000?pkt_size=1316|[select=\'v:1,a:1,d:0\':f=mpegts]udp://239.5.208.109:3000?pkt_size=1316|[select=\'v:2,a:1,d:0\':f=mpegts]udp://239.5.208.109:2000?pkt_size=1316|[select=\'v:3,a:0,d:0\':f=mpegts]udp://239.5.208.109:5000?pkt_size=1316|[select=\'v:4,a:0,d:0\':f=mpegts]udp://239.5.208.109:6000?pkt_size=1316"

Sources:

https://stackoverflow.com/questions/44565968/while-transcoding-to-mpegts-ffmpeg-creates-the-first-frame-after-1-48-seconds-de/49294766

Combining both the fps filter and these muxer options -muxdelay 0 and -avoid_negative_ts disabled pretty much eliminates the A/V sync issues and the gaps the packager reports at the beginning.

Update: You might want to skip this, as there's still an initial DTS gap on UDP input. Still researching.