Yahweasel / libav.js

This is a compilation of the libraries associated with handling audio and video in ffmpeg—libavformat, libavcodec, libavfilter, libavutil, libswresample, and libswscale—for emscripten, and thus the web.
288 stars 18 forks source link

How to add MPEG-TS demux and mpeg2video decoder #21

Closed Luke-HSKim closed 1 year ago

Luke-HSKim commented 1 year ago

I added followings to /configs/default/ffmpeg-config.txt to add MPEG-TS demux and mpeg2video decoder, but it was not successful.

--enable-demuxer=mpegts --enable-decoder=mp1 --enable-decoder=mp2 --enable-decoder=mp3 --enable-decoder=mp3adu --enable-decoder=mp3on4 --enable-decoder=mpeg1video --enable-decoder=mpeg2video

chrome_4tTpIBZ090

How can i make MPEG-TS stream with MPEG2 Video?

Yahweasel commented 1 year ago

It's pretty difficult to guess from this. I've never attempted MPEG-2 per se, though I have done H.263+, which is largely the same, and it works fine. Unfortunately, libav has some stunningly useless error messages, such as the one you're seeing...

My recommended next step would be to check whether the codec it's detecting is actually what it should be, by using avcodec_get_name on the codec ID in the stream.

Luke-HSKim commented 1 year ago

This stream is from DVB-C network.

Input #0, mpegts, from 'D:\Crenova\sources\Tizen3\TiviconConnectTizenWebappPC\libav\capture_das_erste_131s_mpeg2.ts': Duration: 00:02:11.44, start: 3980.301189, bitrate: 6822 kb/s Stream #0:0[0x54]: Data: bin_data ([6][0][0][0] / 0x0006) Stream #0:1[0x65]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Stream #0:20x66: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s Stream #0:30x67: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 192 kb/s (visual impaired) (desc riptions) Stream #0:40x68: Subtitle: dvb_teletext ([6][0][0][0] / 0x0006) Stream #0:50x69: Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) (hearing impaired) Stream #0:60x6a: Audio: ac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 448 kb/s Stream #0:7[0x498]: Unknown: none ([12][0][0][0] / 0x000C) Stream #0:8[0x816]: Unknown: none ([5][0][0][0] / 0x0005) Stream #0:9[0x87b]: Unknown: none ([11][0][0][0] / 0x000B)

When I tried after adding "--enable-demuxer=mpegts", i saw "unsupported codec" or something like this. And I added the rests.

anything I missed except adding to ffmpeg-config.txt?

Yahweasel commented 1 year ago

The question is, what does libav.js say about the codec, not what does ffmpeg say about the codec. I'm trying to narrow down where something's getting lost.

One thing is that you probably don't want to use default as a starting point. You should use one of the configurations that already supports video. They enable swscale, and libav can be pretty dopey with video if you don't have swscale, since that's how it does all pixel format conversions. That being said, it should at least be able to decode, so I don't really know...

Luke-HSKim commented 1 year ago

Thank you for your kind explain!

Followings are all logs until i see alert popup. You can find codec name from libav.js in the middle.

libav-3.11.5.1.2-default.simd.js:708 Readahead input:0 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:32768 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:65536 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:98304 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:131072 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:163840 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:196608 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:229376 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:262144 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:294912 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:327680 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:360448 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:393216 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:425984 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:458752 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:491520 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:524288 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:557056 (32768) demo.html:104 Stream 0: bin_data demo.html:104 Stream 1: mpeg2video demo.html:104 Stream 2: mp3 demo.html:104 Stream 3: mp3 demo.html:104 Stream 4: dvb_teletext demo.html:104 Stream 5: dvb_subtitle demo.html:104 Stream 6: ac3 demo.html:104 Stream 7: none demo.html:104 Stream 8: none demo.html:104 Stream 9: none libav-3.11.5.1.2-default.simd.js:708 Readahead input:0 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:32768 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:65536 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:98304 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:131072 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:163840 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:196608 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:229376 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:262144 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:294912 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:327680 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:360448 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:393216 (32768) libav-3.11.5.1.2-default.simd.js:708 [mpegts @ 0x287100] Packet corrupt (stream = 1, dts = 358295526). put_char @ libav-3.11.5.1.2-default.simd.js:708 write @ libav-3.11.5.1.2-default.simd.js:708 write @ libav-3.11.5.1.2-default.simd.js:708 doWritev @ libav-3.11.5.1.2-default.simd.js:708 _fd_write @ libav-3.11.5.1.2-default.simd.js:708 $func1842 @ libav-3.11.5.1.2-default.simd.wasm:0x1f4332 $func1033 @ libav-3.11.5.1.2-default.simd.wasm:0x10acb2 $func1037 @ libav-3.11.5.1.2-default.simd.wasm:0x10af68 $func593 @ libav-3.11.5.1.2-default.simd.wasm:0x789d3 $func2041 @ libav-3.11.5.1.2-default.simd.wasm:0x21e0cf $func30 @ libav-3.11.5.1.2-default.simd.wasm:0x1ff4 $func1106 @ libav-3.11.5.1.2-default.simd.wasm:0x1279ff $func743 @ libav-3.11.5.1.2-default.simd.wasm:0x9e486 $Bd @ libav-3.11.5.1.2-default.simd.wasm:0x9f93f ret. @ libav-3.11.5.1.2-default.simd.js:708 doRewind @ libav-3.11.5.1.2-default.simd.js:708 (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 handleSleep @ libav-3.11.5.1.2-default.simd.js:708 handleAsync @ libav-3.11.5.1.2-default.simd.js:708 libavjs_wait_reader @ libav-3.11.5.1.2-default.simd.js:708 $func1586 @ libav-3.11.5.1.2-default.simd.wasm:0x1a9163 $func2212 @ libav-3.11.5.1.2-default.simd.wasm:0x23651d $func88 @ libav-3.11.5.1.2-default.simd.wasm:0xa23d $func73 @ libav-3.11.5.1.2-default.simd.wasm:0x7289 $func1208 @ libav-3.11.5.1.2-default.simd.wasm:0x136ffc $func448 @ libav-3.11.5.1.2-default.simd.wasm:0x51c83 $func931 @ libav-3.11.5.1.2-default.simd.wasm:0xe41eb $func1650 @ libav-3.11.5.1.2-default.simd.wasm:0x1b5895 $func1106 @ libav-3.11.5.1.2-default.simd.wasm:0x1276f2 $func743 @ libav-3.11.5.1.2-default.simd.wasm:0x9e486 $Bd @ libav-3.11.5.1.2-default.simd.wasm:0x9f93f ret. @ libav-3.11.5.1.2-default.simd.js:708 ccall @ libav-3.11.5.1.2-default.simd.js:708 (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) (anonymous) @ libav-3.11.5.1.2-default.simd.js:708 Promise.then (async) step @ libav-3.11.5.1.2-default.simd.js:708 libav-3.11.5.1.2-default.simd.js:708 Readahead input:425984 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:458752 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:491520 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:524288 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:557056 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:589824 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:622592 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:655360 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:688128 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:720896 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:753664 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:786432 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:819200 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:851968 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:884736 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:917504 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:950272 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:983040 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:1015808 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:1048576 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:1081344 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:1114112 (32768) libav-3.11.5.1.2-default.simd.js:708 Readahead input:1146880 (32768) libav-3.11.5.1.2-default.simd.js:708 [mpeg2video @ 0x2d3380] Invalid frame dimensions 0x0. p

Yahweasel commented 1 year ago

« libav-3.11.5.1.2-default.simd.js:708 [mpegts @ 0x287100] Packet corrupt (stream = 1, dts = 358295526).»

This is the most suspicious line. Either this is not a valid mpegts stream (mpegps stream?), or something is missing in the demuxing code. What does desktop ffprobe report the file as? Maybe this just isn't the right/most general demuxer.

Luke-HSKim commented 1 year ago

ffprobe.exe capture_das_erste_131s_mpeg2.ts ffprobe version N-91672-gd0b48a9604 Copyright (c) 2007-2018 the FFmpeg developers built with gcc 8.2.1 (GCC) 20180813 configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enabl e-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amr wb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enab le-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 -- enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc - -enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --e nable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth libavutil 56. 19.100 / 56. 19.100 libavcodec 58. 24.101 / 58. 24.101 libavformat 58. 17.103 / 58. 17.103 libavdevice 58. 4.101 / 58. 4.101 libavfilter 7. 26.100 / 7. 26.100 libswscale 5. 2.100 / 5. 2.100 libswresample 3. 2.100 / 3. 2.100 libpostproc 55. 2.100 / 55. 2.100 [mpeg2video @ 0000025ec4127e80] Invalid frame dimensions 0x0. Last message repeated 9 times [mpegts @ 0000025ec40fd680] start time for stream 0 is not set in estimate_timings_from_pts [mpegts @ 0000025ec40fd680] start time for stream 5 is not set in estimate_timings_from_pts [mpegts @ 0000025ec40fd680] PES packet size mismatch Last message repeated 1 times [mpegts @ 0000025ec40fd680] Could not find codec parameters for stream 7 (Unknown: none ([12][0][0][0] / 0x000C)): unkno wn codec Consider increasing the value for the 'analyzeduration' and 'probesize' options [mpegts @ 0000025ec40fd680] Could not find codec parameters for stream 8 (Unknown: none ([5][0][0][0] / 0x0005)): unknow n codec Consider increasing the value for the 'analyzeduration' and 'probesize' options [mpegts @ 0000025ec40fd680] Could not find codec parameters for stream 9 (Unknown: none ([11][0][0][0] / 0x000B)): unkno wn codec Consider increasing the value for the 'analyzeduration' and 'probesize' options Input #0, mpegts, from 'D:\Crenova\sources\Tizen3\TiviconConnectTizenWebappPC\libav\capture_das_erste_131s_mpeg2.ts': Duration: 00:02:11.44, start: 3980.301189, bitrate: 6822 kb/s Program 10400 Program 10401 Program 10402 Program 10403 Program 10404 Program 10405 Program 10406 Program 10407 Program 10408 Program 10432 Program 10433 Program 10434 Program 10451 Program 10452 Program 10453 Program 10454 Program 10455 Program 10456 Program 10457 Program 10458 Program 10459 Program 10465 Program 10466 Program 10467 Program 10468 Program 10469 Program 10470 Program 28106 Stream #0:0[0x54]: Data: bin_data ([6][0][0][0] / 0x0006) Stream #0:1[0x65]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Stream #0:20x66: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 256 kb/s Stream #0:30x67: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 192 kb/s (visual impaired) (desc riptions) Stream #0:40x68: Subtitle: dvb_teletext ([6][0][0][0] / 0x0006) Stream #0:50x69: Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) (hearing impaired) Stream #0:60x6a: Audio: ac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 448 kb/s Stream #0:7[0x498]: Unknown: none ([12][0][0][0] / 0x000C) Stream #0:8[0x816]: Unknown: none ([5][0][0][0] / 0x0005) Stream #0:9[0x87b]: Unknown: none ([11][0][0][0] / 0x000B) Program 28107 Program 28108 Program 28110 Program 28111 Program 28113 Unsupported codec with id 100359 for input stream 0 Unsupported codec with id 94215 for input stream 4 Unsupported codec with id 0 for input stream 7 Unsupported codec with id 0 for input stream 8 Unsupported codec with id 0 for input stream 9

Luke-HSKim commented 1 year ago

it was captured from DVB-C live stream, so probably it has some errors. anyway, it was no problem to playback by vlc or other players.

As I tested other streams, it was stopped always with following error.

libav-3.11.5.1.2-default.simd.js:708 [mpeg2video @ 0x2d3380] Invalid frame dimensions 0x0.

Yahweasel commented 1 year ago

libav.js is not VLC, it's ffmpeg. Do these files work in desktop ffmpeg? If not, the idea of getting them working here is hopeless.

Luke-HSKim commented 1 year ago

i have tried with libavcodec, libavformat, and libavutil, and it could playback correctly.

When I tried at first time, i didn't check logs from ffmpeg libraries, and I checked the logs from pc after your comment. I found there is same "Invalid frame dimensions 0x0.".

Can I ignore some warnings or errors from libav.js?

logs from PC [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpeg2video @ 000001F0180D19C0] Invalid frame dimensions 0x0. [mpegts @ 000001F0180A19C0] Packet corrupt (stream = 1, dts = 358295526). [mpegts @ 000001F0180A19C0] start time for stream 0 is not set in estimate_timings_from_pts [mpegts @ 000001F0180A19C0] start time for stream 5 is not set in estimate_timings_from_pts [mpegts @ 000001F0180A19C0] PES packet size mismatch [mpegts @ 000001F0180A19C0] Packet corrupt (stream = 4, dts = 370027961). [mpegts @ 000001F0180A19C0] PES packet size mismatch [mpegts @ 000001F0180A19C0] Packet corrupt (stream = 6, dts = 370023587). [mpegts @ 000001F0180A19C0] Could not find codec parameters for stream 7 (Unknown: none ([12][0][0][0] / 0x000C)): unknown codec Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options [mpegts @ 000001F0180A19C0] Could not find codec parameters for stream 8 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options [mpegts @ 000001F0180A19C0] Could not find codec parameters for stream 9 (Unknown: none ([11][0][0][0] / 0x000B)): unknown codec Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options [mpegts @ 000001F0180A19C0] Packet corrupt (stream = 1, dts = 358295526). [mpeg2video @ 000001F0180DF240] Invalid mb type in B-frame at 25 27 [mpeg2video @ 000001F0180DF240] Warning MVs not available [mpeg2video @ 000001F0180DF240] concealing 45 DC, 45 AC, 45 MV errors in B frame [mpegts @ 000001F0180A19C0] PES packet size mismatch [mpegts @ 000001F0180A19C0] Packet corrupt (stream = 4, dts = 370027961). [mpegts @ 000001F0180A19C0] PES packet size mismatch [mpegts @ 000001F0180A19C0] Packet corrupt (stream = 6, dts = 370023587). [mpeg2video @ 000001F0180DF240] Warning MVs not available [mpeg2video @ 000001F0180DF240] concealing 225 DC, 225 AC, 225 MV errors in B frame

Yahweasel commented 1 year ago

I actually forgot that I added this, but yes, ff_decode_multi supports ignoring errors. I think the following replacement for the ff_decode_multi line in demo.html might do what you want:

const frames = await libav.ff_decode_multi(
    c, pkt, frame, [vPacket], {
    fin: (res === libav.AVERROR_EOF && vIdx === vPackets.length - 1),
    ignoreErrors: true
    }
);
Luke-HSKim commented 1 year ago

Thank you for your answer! Now it works! But i faced following message. :(

image

Luke-HSKim commented 1 year ago

i could see video after disabling Scaling part and adding to avoid wrong frames like following.

image

But picture is black and white and ratio is completely wrong.

image

Any advice?

Yahweasel commented 1 year ago

See https://github.com/Yahweasel/libav.js/issues/21#issuecomment-1540943086 . This is why the video libavs enable swscale. It does pixel format conversion. What you're looking at is data that's not RGB.

Luke-HSKim commented 1 year ago

I found that frame.channels is not defined, and it causes wrong behavior at scaling functions. I fixed it by converting YCrCb to RGBA and no scaling. Thank you for your kind support!