search

membraneframework / membrane_core

The core of the Membrane Framework, advanced multimedia processing framework
https://membrane.stream
Apache License 2.0
1.27k stars 36 forks source link

`MP4.Demuxer.ISOM` crashes with VP9 video stream #854

Open samrat opened 1 month ago

samrat commented 1 month ago

I got the following crash when trying to demux a mp4 file with a VP9 video stream.

[error] <0.9930.0>/:mp4_demuxer Error occured in Membrane Element:
** (FunctionClauseError) no function clause matching in Membrane.MP4.MovieBox.SampleTableBox.unpack_sample_description/1
    (membrane_mp4_plugin 0.35.1) lib/membrane_mp4/movie_box/sample_table_box.ex:271: Membrane.MP4.MovieBox.SampleTableBox.unpack_sample_description(%{size: 153, fields: %{flags: 0, version: 0, entry_count: 1}, children: [vp09: %{size: 137, content: <<0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 128, 4, 56, 0, 72, 0, 0, 0, 72, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, ...>>, header_size: 8}], header_size: 8})
    (membrane_mp4_plugin 0.35.1) lib/membrane_mp4/movie_box/sample_table_box.ex:240: Membrane.MP4.MovieBox.SampleTableBox.unpack/2
    (membrane_mp4_plugin 0.35.1) lib/membrane_mp4/demuxer/isom/samples_info.ex:144: anonymous fn/1 in Membrane.MP4.Demuxer.ISOM.SamplesInfo.get_samples_info/2
    (elixir 1.17.1) lib/map.ex:257: Map.do_map/2
    (elixir 1.17.1) lib/map.ex:251: Map.new_from_map/2
    (membrane_mp4_plugin 0.35.1) lib/membrane_mp4/demuxer/isom/samples_info.ex:142: Membrane.MP4.Demuxer.ISOM.SamplesInfo.get_samples_info/2
    (membrane_mp4_plugin 0.35.1) lib/membrane_mp4/demuxer/isom.ex:361: Membrane.MP4.Demuxer.ISOM.handle_can_read_mdat_box/2
    (membrane_core 1.1.1) lib/membrane/core/callback_handler.ex:139: Membrane.Core.CallbackHandler.exec_callback/4
    (membrane_core 1.1.1) lib/membrane/core/callback_handler.ex:69: Membrane.Core.CallbackHandler.exec_and_handle_callback/5
    (elixir 1.17.1) lib/enum.ex:2531: Enum."-reduce/3-lists^foldl/2-0-"/3
    (membrane_core 1.1.1) lib/membrane/core/element/buffer_controller.ex:81: Membrane.Core.Element.BufferController.do_handle_incoming_buffers/4
    (membrane_core 1.1.1) lib/membrane/core/element.ex:231: Membrane.Core.Element.handle_info/2
    (stdlib 5.1.1) gen_server.erl:1077: :gen_server.try_handle_info/3
    (stdlib 5.1.1) gen_server.erl:1165: :gen_server.handle_msg/6
    (stdlib 5.1.1) proc_lib.erl:241: :proc_lib.init_p_do_apply/3

mp4 file ffprobe

ffprobe version 7.0.1 Copyright (c) 2007-2024 the FFmpeg developers
  built with Apple clang version 15.0.0 (clang-1500.3.9.4)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/7.0.1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'mp4_plugin_crash.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf61.1.100
  Duration: 00:09:11.98, start: 0.000000, bitrate: 4300 kb/s
  Stream #0:0[0x1](und): Video: vp9 (Profile 0) (vp09 / 0x39307076), yuv420p(tv, bt709), 1920x1080, 4167 kb/s, 25 fps, 25 tbr, 16k tbn (default)
      Metadata:
        handler_name    : ISO Media file produced by Google Inc. Created on: 08/14/2024.
        vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
      Metadata:
        handler_name    : ISO Media file produced by Google Inc.
        vendor_id       : [0][0][0][0]
samrat commented 1 month ago

Looks like there is a plugin for decoding VP9 streams(https://github.com/membraneframework/membrane_vpx_plugin). Would the fix be for the MP4 plugin to use that plugin?

Or would I need to add this to my application's pipeline?

mat-hek commented 1 month ago

Hi @samrat, VPx in MP4 is generally not widely supported, and Membrane doesn't support it either. The fix, or rather a feature, would be to add support for extracting VPx from MP4 in Membrane.MP4.Demuxer.ISOM. The VPx plugin is about encoding and decoding, so it's a different thing.