How to set demuxer options

[reinhrst]

I was wondering if it's possible to set demuxer options. Specifically I would like to increase max_packet_size for the mpegts demuxer: https://ffmpeg.org/ffmpeg-formats.html#mpegts.

With the current default value of 204800 bytes, frames that need more than 204800 get split when av_read_frame() is called.

(I guess patching ffmpeg to have another default value before compiling is one solution, but is there a better way?)

[Yahweasel]

At the moment it's not possible in ff_init_demuxer_file, but only because nobody's asked for it until now :) . I can add it, but no promises on the timeline. In the interim, ff_init_demuxer_file is just a frontend for the lower-level libav functions, and you can always use them directly. See https://github.com/Yahweasel/libav.js/blob/9cce05ba80b9d7ea763538efd62e0f74dc18ff12/post.in.js#L1008 and, for an example of how passing options is done through JavaScript, see https://github.com/Yahweasel/libav.js/blob/9cce05ba80b9d7ea763538efd62e0f74dc18ff12/post.in.js#L671 .

[reinhrst]

Ah, that moves me in the right direction! I'm actually using the lower level functions already (basically using the code from /tests/tests/500-remux.js). It seems I should be able to use these options on avformat_open_input_js as third paramater. I don't quite have the right format yet, but at least back on track :).

Something like this should do it...

  const options = await libav.av_dict_set_js(0, "max_packet_size", "1000000", 0)
  const ifmt_ctx = await libav.avformat_open_input_js(in_filename, 0, options);

(but for now gives a "memory access out of bounds" error, so some more tweaking is needed; will do so soon)

[Yahweasel]

As far as I know that should work. Unfortunately, asynchrony can make figuring out where such errors actually happen... difficult. You may want to try loading with noworker to see if you get a clearer stack frame.

[reinhrst]

Just reporting on my progress...

With {noworker: true} I did get a clearer stack frame.

dlmalloc.c:4786 Uncaught (in promise) RuntimeError: memory access out of bounds
    at libav-4.5.6.0-behave.dbg.simd.wasm.dlfree (dlmalloc.c:4786:24)
    at libav-4.5.6.0-behave.dbg.simd.wasm.av_freep (mem.c:241:5)
    at libav-4.5.6.0-behave.dbg.simd.wasm.av_dict_free (dict.c:234:5)
    at libav-4.5.6.0-behave.dbg.simd.wasm.avformat_open_input (demux.c:350:9)
    at libav-4.5.6.0-behave.dbg.simd.wasm.avformat_open_input_js (bindings.c:516:15)
    at ret.<computed> (libav-4.5.6.0-behave.dbg.simd.js:5109:35)
    at Object.doRewind (libav-4.5.6.0-behave.dbg.simd.js:5209:16)
    at libav-4.5.6.0-behave.dbg.simd.js:5235:47

So as far as I can see, this crashes on freeing the options (specifically av_freep(pm)).

As a separate issue (for me); if I comment out this line, the max_packet_size is still not passed on to the demuxer.

Hardcoding a higher default packet size in ffmpeg works (but is hardly a nice solution).

[reinhrst]

In the interest of moving forward with my project (and not just digging deeper and deeper), for now I'm happy to hardcode the higher default value in ffmpeg (adding the following lines to ffmpeg.diff):

diff --git a/libavformat/mpegts.c b/libavformat/mpegts.c
index d97702fcd7..f08a240785 100644
--- a/libavformat/mpegts.c
+++ b/libavformat/mpegts.c
@@ -202,7 +202,7 @@ static const AVOption options[] = {
     {"skip_clear", "skip clearing programs", offsetof(MpegTSContext, skip_clear), AV_OPT_TYPE_BOOL,
      {.i64 = 0}, 0, 1, 0 },
     {"max_packet_size", "maximum size of emitted packet", offsetof(MpegTSContext, max_packet_size), AV_OPT_TYPE_INT,
-     {.i64 = 204800}, 1, INT_MAX/2, AV_OPT_FLAG_DECODING_PARAM },
+     {.i64 = 20480000}, 1, INT_MAX/2, AV_OPT_FLAG_DECODING_PARAM },
     { NULL },
 };

I may revisit this later, and if so may post more info here or in a new issue.

[reinhrst]

Considering your answer to #35 (and considering that the errors I got there, and the error I got in #35 are the same), is it possible that the cause is the same too (for the following code)?

  const options = await libav.av_dict_set_js(0, "max_packet_size", "1000000", 0)
  const ifmt_ctx = await libav.avformat_open_input_js(in_filename, 0, options);

Looking at bindings.c, it indeed seems that libav.avformat_open_input_js expects an AVDictionary**, whereas av_dict_set_js returns an AVDictionary*.

[reinhrst]

Update: Indeed, changing avformat_open_input_js to receive an AVDictionary * fixes the error (and also makes the max_packet_size option be picked up by the demuxer 🎉).

@@ -510,10 +510,12 @@ AVFormatContext *avformat_alloc_output_context2_js(AVOutputFormat *oformat,
 }

 AVFormatContext *avformat_open_input_js(const char *url, AVInputFormat *fmt,
-    AVDictionary **options)
+    AVDictionary *options)
 {
     AVFormatContext *ret = NULL;
-    int err = avformat_open_input(&ret, url, fmt, options);
+    AVDictionary** options_p = &options;
+    int err = avformat_open_input(&ret, url, fmt, options_p);
     if (err < 0)
         fprintf(stderr, "[avformat_open_input_js] %s\n", av_err2str(err));
     return ret;

I don't have a good enough view of the big picture to know if this is a fix, or is there another approach?

[Yahweasel]

Damn, you're absolutely right. It's easy to miss things like this, as I've never actually used a dictionary in the open options @_@ . If you make that a PR, I'll merge it :+1:

[reinhrst]

Will do. Should I do the same for avio_open2_js() in the same PR?