Latency issue possibly caused by avformat_find_stream_info

[glennolsen]

Live streaming through Nginx to avconv to bmdplay to Quad SDI card.

Was getting 2x a latency value (5-6s). Once through avconv and once through bmdplay. I was able to almost eliminate the latency through avconv with explicitly setting framerate (since I know it) and also setting the max time that avformat_find_stream_info looks for stream information.

-r 30 -probesize 750000 -analyzeduration 750000

full command:

avconv -r 30 -max_delay 1000 -probesize 750000 -analyzeduration 750000 -fflags nobuffer -i rtmp://localhost:1935/AV2015Local/stream2 -f nut -strict experimental -c:v copy -c:a pcm_s16le - | /home/vidcontrol/Builds/'Blackmagic DeckLink SDK 10.4.1'/Linux/Samples/bmdtools-master/bmdplay -m 10 -C 1 -f pipe:0

I am looking for something similar in bmdplay. Even better, since my format is fixed (I control the encoding and the RTMP streaming), I would like to be able to provide all of these hints up front so that I can reduce start up latency through bmdplay.

In fact, there is a note in the libav code about doing just this

from libav/libavformat/avformat.h @ ln 1445

• @todo Let the user decide somehow what information is needed so that
  • we do not waste time getting stuff the user does not need. _/ int avformat_find_stream_info(AVFormatContext ic, AVDictionary _options);

avconv using avformat looks for these supplied by user

nb_audio_sample_rate nb_audio_channels nb_frame_rates nb_frame_sizes nb_frame_pix_fmts

then picks a codec (decoder) which will have it's own set of parameters and then uses avformat_find_stream_info to fill in the blanks.

In bmdplay I think it would be possible to set these after here:

378 avformat_find_stream_info(ic, NULL);

ic->max_delay = 1000; ic->framerate = 30; ic->flags |= AVFMT_FLAG_NOBUFFER; ic->probesize = 750000; ic->max_analyze_duration = 750000; ic->fps_probe_size = ?????; ic->strict_std_compliance = true;

Of course the right way to do this is through the command line args.

Am I on the right track?

[glennolsen]

I mistakenly wrote above that this code should be after but it should be before.

Using the following test code I was able to reduce the latency in bmdplay by ~30%:

ic->max_delay = 1000;
ic->flags &= AVFMT_FLAG_NOBUFFER;
ic->probesize = 1000000;
ic->max_analyze_duration = 1000000;
ic->fps_probe_size = 30;
for (int i = 0; i < ic->nb_streams; i++) {
    AVStream *st = ic->streams[i];
    st->avg_frame_rate.num = 30;
    st->avg_frame_rate.den = 1;
}

[lu-zero]

Thanks for reminding me, I'll add soon a nolatency option.

[lu-zero]

I'll add a no-latency option to skip that part completely soon, thank for reminding me!

[glennolsen]

Luca,

Thanks. Not pushing but our event is in three weeks. Which part would you skip? The function call I noted or somewhere else?

Thanks again. 

Glenn

— Sent from Mailbox

On Thu, Jun 25, 2015 at 12:28 AM, Luca Barbato notifications@github.com wrote:

I'll add a no-latency option to skip that part completely soon, thank for

reminding me!

Reply to this email directly or view it on GitHub: https://github.com/lu-zero/bmdtools/issues/58#issuecomment-115109107

[lu-zero]

In theory all the information should be already provided by NUT (that's why I'm using it), in theory the find_stream_info call should be a no-op.

If it is not currently, dropping it shouldn't impact the rest as long NUT is used, possibly the -syncpoints none variant.

[glennolsen]

Got it. I'll give it a try

— Sent from Mailbox

On Thu, Jun 25, 2015 at 12:54 AM, Luca Barbato notifications@github.com wrote:

In theory all the information should be already provided by NUT (that's why I'm using it), in theory the find_stream_info call should be a no-op.

If it is not currently, dropping it shouldn't impact the rest as long NUT is used, possibly the -syncpoints none variant.

Reply to this email directly or view it on GitHub: https://github.com/lu-zero/bmdtools/issues/58#issuecomment-115114771

[lu-zero]

Looks like there is an audio parameter missing, so it isn't just dropping that call.

You are doing -c:v copy, decode it with -c:v rawvideo or -c:v v210 if you want 10bits output.

[glennolsen]

Luca,

A question, the video is h.264 encoded by Teradek Cube, 720p@30fps. I am trying to output through bmdplay and Quadlink SDI card at 1080i@59fps. Having a very difficult time getting scaling to work. Is this related to using v:copy?

avconv -r 30 -max_delay 1000 -probesize 500000 -analyzeduration 3000000 -fflags nobuffer -i rtmp://localhost:1935/AV2015Local/stream2 -f nut -strict experimental -c:v copy -c:a pcm_s16le - | /home/vidcontrol/Builds/'Blackmagic DeckLink SDK 10.4.1'/Linux/Samples/bmdtools-master/bmdplay -m 12 -C 1 -f pipe:0

I get a 720p region in the upper left corner. Thoughts? Suggestions?

I am not too familiar with Avconv and this may be better posted over there.

Glenn

[haakonnessjoen]

You are not doing any scaling. bmplay does not scale for you. You do it with avconv. Add a -s 1920x1080 or use -vf scale

On 11 July 2015 at 08:39, glennolsen notifications@github.com wrote:

Luca,

A question, the video is h.264 encoded by Teradek Cube, 720p@30fps. I am trying to output through bmdplay and Quadlink SDI card at 1080i@59fps. Having a very difficult time getting scaling to work. Is this related to using v:copy?

avconv -r 30 -max_delay 1000 -probesize 500000 -analyzeduration 3000000 -fflags nobuffer -i rtmp://localhost:1935/AV2015Local/stream2 -f nut -strict experimental -c:v copy -c:a pcm_s16le - | /home/vidcontrol/Builds/'Blackmagic DeckLink SDK 10.4.1'/Linux/Samples/bmdtools-master/bmdplay -m 12 -C 1 -f pipe:0

I get a 720p region in the upper left corner. Thoughts? Suggestions?

I am not too familiar with Avconv and this may be better posted over there.

Glenn

— Reply to this email directly or view it on GitHub https://github.com/lu-zero/bmdtools/issues/58#issuecomment-120587313.

Håkon

[glennolsen]

Thanks, I pasted the wrong command I have tried both. Turns out that it is my error. I am streaming 720p and the Blackmagic card is scaling the SDI output to 1080i because that is how I configured it in Desktop Video Utility. Looks good for now. On to the next issue that I will post in another thread.

Thanks again.

Glenn