search

dmlc / decord

An efficient video loader for deep learning with smart shuffling that's super easy to digest
Apache License 2.0
1.85k stars 160 forks source link

Extremely slow accurate seek #111

Open trifle opened 3 years ago

trifle commented 3 years ago

Hi, I've hit an unexpected regression with accurate seeks - they seem about one order of magniture too slow. This is with a CUDA-enabled manual install from current git HEAD:

python3 decord/tests/benchmark/bench_decord.py --file test.mp4
15000  frames, elapsed time for sequential read:  9.465791463851929
300  frames, elapsed time for random access(not accurate):  2.511744737625122
300  frames, elapsed time for random access(accurate):  416.69994831085205

The machine (16 true core, nvme SSD) is neither CPU nor IO limited during the benchmark. CPU load is ~8 (out of 16), so half the cores are idle. The file is a 50MB h.264 video encoded by ffmpeg.

Results are slightly faster for random but even worse for accurate seeks when running on GPU:

root@428b7218513b:/orb# python3 decord/tests/benchmark/bench_decord.py --file test.mp4 --gpu 0
15000  frames, elapsed time for sequential read:  5.069800615310669
300  frames, elapsed time for random access(not accurate):  0.4220912456512451
300  frames, elapsed time for random access(accurate):  535.492870092392

This is on a RTX 2070 super.

Other specs:

# python3 -V
Python 3.6.9
# pip3 list | grep decord
decord                 0.4.2

Comparison benchmarks (don't have pyav, so for opencv only):

# python3 decord/tests/benchmark/bench_
bench_decord.py  bench_opencv.py  bench_pyav.py
root@428b7218513b:/orb# python3 decord/tests/benchmark/bench_opencv.py --file test.mp4
15000  frames. Elapsed time for sequential read:  5.646612882614136
300  frames, elapsed time for random access(not accurate):  13.583254337310791

If there's anything else I can do to help debugging, let me know!

trifle commented 3 years ago

Just to substantiate: Another benchmark run from a smaller (SD) file with same duration (10m):

# python3 decord/tests/benchmark/bench_decord.py --file test2.mp4
14945  frames, elapsed time for sequential read:  9.622890949249268
300  frames, elapsed time for random access(not accurate):  2.366671323776245
300  frames, elapsed time for random access(accurate):  285.0761547088623
trifle commented 3 years ago

And some info on the video file:

ffprobe version 4.3.1-0york0~18.04 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/usr --extra-version='0york0~18.04' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libzimg --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
  Duration: 00:10:00.00, start: 0.000000, bitrate: 693 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x576 [SAR 16:11 DAR 20:11], 591 kb/s, SAR 64:45 DAR 16:9, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(deu): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 97 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
trifle commented 3 years ago

On further investigation, it seems the slow iteration is due to the video files. I'll leave this open in case there is some helpful input and investigate further.

trifle commented 3 years ago

My intuition is that these video files might lack some sort of header information, which would require decord to scan the entire file on each individual indexing step. I.e., in order to get frames [1,5,10], the library would need to scan the file thrice.

I've solved the problem by manually iterating over all frames and skipping those not in the batch set.

zhreshold commented 3 years ago

Can you print the keyframes of the testing video you have? You can call vr.get_key_indices(). Another potential cause is the new RTX 2000 series with tensor cores. I haven't really tried the decoding with the newer cards, and not exactly sure the difference in cuviddec since I am using header definition from older cuda drive which drivers have been updated.

trifle commented 3 years ago

Thanks for the pointer, I'll investigate that. Ffprobe listed the keyframes as approx. 60 apart, IIRC.

As far as the RTX 2000 cards go, I think I read about some minor improvements in encoding quality, but the decoding seemed pretty stable since multiple generations.

By the way, a side effect of my manual iteration is that decord seems to load all frames in memory - so it's consuming ~40GB of ram for a 200MB video file. So there is some incentive for me to get indexing working.

trifle commented 3 years ago

OK, for documentation, here are some further details. It appears the keyframes in the first testfile are extremely widely spaced.

In [2]: vr = VideoReader("test.mp4")
In [3]: vr.get_key_indices()[:10]
Out[3]: [0, 250, 500, 750, 1000, 1250, 1500, 1750, 2000, 2250]

In [4]: vr = VideoReader("test2.mp4")
In [5]: vr.get_key_indices()[:10]
Out[5]: [0, 190, 440, 690, 756, 794, 842, 879, 973, 1059]

By the way, I could get around the memory bloat by decoding the files on the GPU - even though the GPU decoder is a bit slower than the CPU (probably due to copying - my code isn't optimized towards using the zero copy facilities from decord).

mark-aai commented 3 years ago

I had a similar problem in simply doing something like this on cpu on ubuntu:

for i in range(10): frame = vr[i*100].asnumpy()

Each line got slower as the loop progressed. The problem went away when I reverted from 0.4.2 back to 0.4.0

trifle commented 3 years ago

Hi @mark-aai, thanks for the info, that's exactly the pathological case I am experiencing! Sounds like this could be a regression.

[edit] Although, looking at the commits, I don't see anything in particular that stands out between 0.4.0 and 0.4.2 ... Especially since you're decoding on the CPU and not on GPU.

mark-aai commented 3 years ago

Hi @trifle , the only thing I could add is I used pip to install both times. First "pip install decord", which installed 0.4.2, and then later as soon as I ran "pip install decord==0.4.0" the problem went away. I tried this because I checked what version was running fine on a different machine, and it was 0.4.0.

frankier commented 3 years ago

Here's another example using CPU and self-contained. After following compilation instructions:

$ wget https://youtube-dl.org/downloads/latest/youtube-dl
$ chmod +x ./youtube-dl
$ ./youtube-dl -o breakingnews.mp4 -f 'bestvideo' 'https://www.youtube.com/watch?v=9U4Ha9HQvMo'
$ python3 ../tests/benchmark/bench_decord.py --file breakingnews.py
3645  frames, elapsed time for sequential read:  2.2808821201324463
300  frames, elapsed time for random access(not accurate):  1.709573745727539
300  frames, elapsed time for random access(accurate):  180.16735744476318

For this benchmark, the accurate seeking seems to often take approximately the runtime of the video.

The problem probably stems from https://github.com/dmlc/decord/pull/78 . Apparently that works fine for some video formats but not for others. Is there any way we can determine which formats can't accurately seek to a keyframe, and only use this aggressive/slow strategy of seeking to 0 and then double checking in those cases? It seems that one of these is the most likely culprit. If it's not possible to know beforehand, we could let the user choose the strategy. Either:

  1. Trust seek is accurate (current inaccurate seek)
  2. Trust seek to keyframe is accurate
  3. Seek to keyframe then check
  4. Seek to 0 then seek to keyframe then check (current accurate)

From the benchmarks in that PR it's clear that the new strategy doesn't necessarily slow down all formats in the same way. On the other hand, it's probably the case that many formats can accurately seek to keyframes.

By the way, my problem here that that now get_batch(...) is extremely slow. Since this is the only batching method there should be a fast way of doing this I think.

frankier commented 3 years ago

@innerlee @JoannaLXY Any thoughts about this?

innerlee commented 3 years ago

What's the speed on v0.4.1?

frankier commented 3 years ago

Okay I've gone ahead and bisected it manually and for the same example which anyone can youtube-dl themselves, it seems I pointed to the wrong culprit. (Sorry @innerlee & @JoannaLXY!)

449a9ff

3645  frames, elapsed time for sequential read:  2.0453379154205322
300  frames, elapsed time for random access(not accurate):  1.9726817607879639
300  frames, elapsed time for random access(accurate):  8.188988208770752

dfab508

3645  frames, elapsed time for sequential read:  2.131503105163574
300  frames, elapsed time for random access(not accurate):  2.058363199234009
300  frames, elapsed time for random access(accurate):  9.104778289794922

So it's a much smaller perf regression than the one we're encountering (and worth it if it improves accuracy). I'll carry on bisecting and report back where the regression happened. I would be good to have some simple tests to avoid such big regressions even making it in.

frankier commented 3 years ago

Okay the problem was introduced here: https://github.com/dmlc/decord/commit/7b6c0e9a4d404087fc1fcede58b3720bb67d425b

584e5f8

3645  frames, elapsed time for sequential read:  2.1434860229492188
300  frames, elapsed time for random access(not accurate):  2.1971733570098877
300  frames, elapsed time for random access(accurate):  9.710346460342407

7b6c0e9

3645  frames, elapsed time for sequential read:  2.093999147415161
300  frames, elapsed time for random access(not accurate):  1.8065061569213867
300  frames, elapsed time for random access(accurate):  185.38359928131104
innerlee commented 3 years ago

For random access, if you can get the indices ahead of time, then sorting and accessing sequentially will improve speed.

See code here https://github.com/open-mmlab/mmaction2/blob/master/mmaction/datasets/pipelines/loading.py#L971-L976

frankier commented 3 years ago

Yeah that's a good tip (and quite similar to what get_batch(...) does internally I think) but it seems the problem here is probably the switch to frame number based seeking from PTS based seeking in 7b6c0e9 . I would hazzard a guess that this is fine for some formats but really slow for others. I would suppose that there's enough info for @zhreshold to come up with some possible next steps anyway.

frankier commented 3 years ago

However v0.4.0 is the last known good version rather than v0.4.1 due to https://github.com/dmlc/decord/issues/100

innerlee commented 3 years ago

That was mostly because of bad video files, which can be fixed on the user side. We used v0.4.1 on the whole kinetics 700 (re-encoded because of resize), and found no issue

zhreshold commented 3 years ago

Okay the problem was introduced here: 7b6c0e9

584e5f8

3645  frames, elapsed time for sequential read:  2.1434860229492188
300  frames, elapsed time for random access(not accurate):  2.1971733570098877
300  frames, elapsed time for random access(accurate):  9.710346460342407

7b6c0e9

3645  frames, elapsed time for sequential read:  2.093999147415161
300  frames, elapsed time for random access(not accurate):  1.8065061569213867
300  frames, elapsed time for random access(accurate):  185.38359928131104

@frankier interesting that frame based seeking is way slower in some cases, I am a bit confused now since there's no documentation of ffmpeg to detail the implementation strategy

zhreshold commented 3 years ago

@frankier and also my own test using the same video you have provided shown a very different result:

Results on mac with (Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz)

master/HEAD

3645  frames, elapsed time for sequential read:  2.934558153152466
300  frames, elapsed time for random access(not accurate):  2.2726590633392334
300  frames, elapsed time for random access(accurate):  12.857761144638062

revert 7b6c0e9:

3645  frames, elapsed time for sequential read:  3.015800714492798
300  frames, elapsed time for random access(not accurate):  2.3213109970092773
300  frames, elapsed time for random access(accurate):  13.505981922149658

There's no differece before/after 7b6c0e9 which is weird. It's possibly due to the ffmpeg version? mine is:

libdecord.dylib:
    @rpath/libdecord.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/opt/ffmpeg/lib/libavformat.58.dylib (compatibility version 58.0.0, current version 58.12.100)
    /usr/local/opt/ffmpeg/lib/libavfilter.7.dylib (compatibility version 7.0.0, current version 7.16.100)
    /usr/local/opt/ffmpeg/lib/libavcodec.58.dylib (compatibility version 58.0.0, current version 58.18.100)
    /usr/local/opt/ffmpeg/lib/libavutil.56.dylib (compatibility version 56.0.0, current version 56.14.100)
    /usr/local/opt/ffmpeg/lib/libavdevice.58.dylib (compatibility version 58.0.0, current version 58.3.100)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
zhreshold commented 3 years ago

Anyway I am reverting the frame based seeking to ts based in #115, I will keep this thread open as I haven't found the root cause of the issue and the issue seems to be more complicated and not well reproducible.

frankier commented 3 years ago

I am using Debian Bullseye. It looks like I have the same version of the relevant FFMPEG/lib although am missing libavdevice. Here is the full info:

frankier ~/.../decord/build % ldd libdecord.so                        (7b6c0e9)
    linux-vdso.so.1 (0x00007ffefcfa2000)
    libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007f15fd55a000)
    libavfilter.so.7 => /lib/x86_64-linux-gnu/libavfilter.so.7 (0x00007f15fd14e000)
    libavcodec.so.58 => /lib/x86_64-linux-gnu/libavcodec.so.58 (0x00007f15fbb96000)
    libavutil.so.56 => /lib/x86_64-linux-gnu/libavutil.so.56 (0x00007f15fb8d5000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f15fb708000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f15fb5c2000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f15fb5a8000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f15fb3e3000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f15fd8b1000)
    libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f15fb235000)
    libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f15fb222000)
    libgme.so.0 => /lib/x86_64-linux-gnu/libgme.so.0 (0x00007f15fb1d3000)
    libopenmpt.so.0 => /lib/x86_64-linux-gnu/libopenmpt.so.0 (0x00007f15fafe7000)
    libchromaprint.so.1 => /lib/x86_64-linux-gnu/libchromaprint.so.1 (0x00007f15fafd1000)
    libbluray.so.2 => /lib/x86_64-linux-gnu/libbluray.so.2 (0x00007f15faf86000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f15faf69000)
    libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f15fad9f000)
    librabbitmq.so.4 => /lib/x86_64-linux-gnu/librabbitmq.so.4 (0x00007f15fad87000)
    libssh-gcrypt.so.4 => /lib/x86_64-linux-gnu/libssh-gcrypt.so.4 (0x00007f15facf6000)
    libzmq.so.5 => /lib/x86_64-linux-gnu/libzmq.so.5 (0x00007f15fac39000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f15fac17000)
    libswscale.so.5 => /lib/x86_64-linux-gnu/libswscale.so.5 (0x00007f15fab7e000)
    libpostproc.so.55 => /lib/x86_64-linux-gnu/libpostproc.so.55 (0x00007f15fab5b000)
    libswresample.so.3 => /lib/x86_64-linux-gnu/libswresample.so.3 (0x00007f15fab39000)
    libpocketsphinx.so.3 => /lib/x86_64-linux-gnu/libpocketsphinx.so.3 (0x00007f15faaec000)
    libsphinxbase.so.3 => /lib/x86_64-linux-gnu/libsphinxbase.so.3 (0x00007f15faaa6000)
    libbs2b.so.0 => /lib/x86_64-linux-gnu/libbs2b.so.0 (0x00007f15faa9e000)
    liblilv-0.so.0 => /lib/x86_64-linux-gnu/liblilv-0.so.0 (0x00007f15faa82000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f15faa7c000)
    librubberband.so.2 => /lib/x86_64-linux-gnu/librubberband.so.2 (0x00007f15faa41000)
    libmysofa.so.1 => /lib/x86_64-linux-gnu/libmysofa.so.1 (0x00007f15faa2e000)
    libflite_cmu_us_awb.so.1 => /lib/x86_64-linux-gnu/libflite_cmu_us_awb.so.1 (0x00007f15fa637000)
    libflite_cmu_us_kal.so.1 => /lib/x86_64-linux-gnu/libflite_cmu_us_kal.so.1 (0x00007f15fa4d1000)
    libflite_cmu_us_kal16.so.1 => /lib/x86_64-linux-gnu/libflite_cmu_us_kal16.so.1 (0x00007f15fa0f7000)
    libflite_cmu_us_rms.so.1 => /lib/x86_64-linux-gnu/libflite_cmu_us_rms.so.1 (0x00007f15f9c5c000)
    libflite_cmu_us_slt.so.1 => /lib/x86_64-linux-gnu/libflite_cmu_us_slt.so.1 (0x00007f15f9863000)
    libflite.so.1 => /lib/x86_64-linux-gnu/libflite.so.1 (0x00007f15f9823000)
    libfribidi.so.0 => /lib/x86_64-linux-gnu/libfribidi.so.0 (0x00007f15f9807000)
    libass.so.9 => /lib/x86_64-linux-gnu/libass.so.9 (0x00007f15f97d2000)
    libva.so.2 => /lib/x86_64-linux-gnu/libva.so.2 (0x00007f15f97a5000)
    libvidstab.so.1.1 => /lib/x86_64-linux-gnu/libvidstab.so.1.1 (0x00007f15f978f000)
    libOpenCL.so.1 => /lib/x86_64-linux-gnu/libOpenCL.so.1 (0x00007f15f9782000)
    libfontconfig.so.1 => /lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f15f973a000)
    libfreetype.so.6 => /lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f15f9677000)
    libmfx.so.1 => /lib/x86_64-linux-gnu/libmfx.so.1 (0x00007f15f9669000)
    libvpx.so.6 => /lib/x86_64-linux-gnu/libvpx.so.6 (0x00007f15f9432000)
    libwebpmux.so.3 => /lib/x86_64-linux-gnu/libwebpmux.so.3 (0x00007f15f9426000)
    libwebp.so.6 => /lib/x86_64-linux-gnu/libwebp.so.6 (0x00007f15f93b9000)
    liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f15f9390000)
    libdav1d.so.4 => /lib/x86_64-linux-gnu/libdav1d.so.4 (0x00007f15f925c000)
    librsvg-2.so.2 => /lib/x86_64-linux-gnu/librsvg-2.so.2 (0x00007f15f885c000)
    libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007f15f8802000)
    libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f15f86d3000)
    libcairo.so.2 => /lib/x86_64-linux-gnu/libcairo.so.2 (0x00007f15f85b1000)
    libzvbi.so.0 => /lib/x86_64-linux-gnu/libzvbi.so.0 (0x00007f15f8521000)
    libsnappy.so.1 => /lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007f15f8516000)
    libaom.so.0 => /lib/x86_64-linux-gnu/libaom.so.0 (0x00007f15f807e000)
    libcodec2.so.0.9 => /lib/x86_64-linux-gnu/libcodec2.so.0.9 (0x00007f15f7299000)
    libgsm.so.1 => /lib/x86_64-linux-gnu/libgsm.so.1 (0x00007f15f7289000)
    libmp3lame.so.0 => /lib/x86_64-linux-gnu/libmp3lame.so.0 (0x00007f15f720f000)
    libopenjp2.so.7 => /lib/x86_64-linux-gnu/libopenjp2.so.7 (0x00007f15f71b8000)
    libopus.so.0 => /lib/x86_64-linux-gnu/libopus.so.0 (0x00007f15f715d000)
    libshine.so.3 => /lib/x86_64-linux-gnu/libshine.so.3 (0x00007f15f6f52000)
    libspeex.so.1 => /lib/x86_64-linux-gnu/libspeex.so.1 (0x00007f15f6f37000)
    libtheoraenc.so.1 => /lib/x86_64-linux-gnu/libtheoraenc.so.1 (0x00007f15f6efa000)
    libtheoradec.so.1 => /lib/x86_64-linux-gnu/libtheoradec.so.1 (0x00007f15f6ed8000)
    libtwolame.so.0 => /lib/x86_64-linux-gnu/libtwolame.so.0 (0x00007f15f6eb3000)
    libvorbis.so.0 => /lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007f15f6e86000)
    libvorbisenc.so.2 => /lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007f15f6ddb000)
    libwavpack.so.1 => /lib/x86_64-linux-gnu/libwavpack.so.1 (0x00007f15f6daf000)
    libx264.so.160 => /lib/x86_64-linux-gnu/libx264.so.160 (0x00007f15f6aed000)
    libx265.so.192 => /lib/x86_64-linux-gnu/libx265.so.192 (0x00007f15f5b6e000)
    libxvidcore.so.4 => /lib/x86_64-linux-gnu/libxvidcore.so.4 (0x00007f15f5a5b000)
    libva-drm.so.2 => /lib/x86_64-linux-gnu/libva-drm.so.2 (0x00007f15f5a56000)
    libva-x11.so.2 => /lib/x86_64-linux-gnu/libva-x11.so.2 (0x00007f15f5a4e000)
    libvdpau.so.1 => /lib/x86_64-linux-gnu/libvdpau.so.1 (0x00007f15f5a48000)
    libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007f15f5903000)
    libdrm.so.2 => /lib/x86_64-linux-gnu/libdrm.so.2 (0x00007f15f58ef000)
    libicuuc.so.67 => /lib/x86_64-linux-gnu/libicuuc.so.67 (0x00007f15f5707000)
    libmpg123.so.0 => /lib/x86_64-linux-gnu/libmpg123.so.0 (0x00007f15f56a3000)
    libvorbisfile.so.3 => /lib/x86_64-linux-gnu/libvorbisfile.so.3 (0x00007f15f5698000)
    libudfread.so.0 => /lib/x86_64-linux-gnu/libudfread.so.0 (0x00007f15f568b000)
    libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f15f5557000)
    libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f15f5536000)
    libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f15f53b4000)
    libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f15f539e000)
    libnettle.so.8 => /lib/x86_64-linux-gnu/libnettle.so.8 (0x00007f15f535e000)
    libhogweed.so.6 => /lib/x86_64-linux-gnu/libhogweed.so.6 (0x00007f15f5313000)
    libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f15f5292000)
    libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f15f5200000)
    libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f15f4f0c000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f15f4f01000)
    libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007f15f4de1000)
    libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f15f4db9000)
    libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f15f4d66000)
    libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f15f4d4c000)
    libsodium.so.23 => /lib/x86_64-linux-gnu/libsodium.so.23 (0x00007f15f4cf2000)
    libpgm-5.3.so.0 => /lib/x86_64-linux-gnu/libpgm-5.3.so.0 (0x00007f15f4ca2000)
    libnorm.so.1 => /lib/x86_64-linux-gnu/libnorm.so.1 (0x00007f15f4b49000)
    libsoxr.so.0 => /lib/x86_64-linux-gnu/libsoxr.so.0 (0x00007f15f4adc000)
    libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x00007f15f4a6f000)
    liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3 (0x00007f15f43cc000)
    libserd-0.so.0 => /lib/x86_64-linux-gnu/libserd-0.so.0 (0x00007f15f43b3000)
    libsord-0.so.0 => /lib/x86_64-linux-gnu/libsord-0.so.0 (0x00007f15f43a7000)
    libsratom-0.so.0 => /lib/x86_64-linux-gnu/libsratom-0.so.0 (0x00007f15f4399000)
    libsamplerate.so.0 => /lib/x86_64-linux-gnu/libsamplerate.so.0 (0x00007f15f402d000)
    libfftw3.so.3 => /lib/x86_64-linux-gnu/libfftw3.so.3 (0x00007f15f3e26000)
    libflite_usenglish.so.1 => /lib/x86_64-linux-gnu/libflite_usenglish.so.1 (0x00007f15f3dfa000)
    libflite_cmulex.so.1 => /lib/x86_64-linux-gnu/libflite_cmulex.so.1 (0x00007f15f3d63000)
    libasound.so.2 => /lib/x86_64-linux-gnu/libasound.so.2 (0x00007f15f3c65000)
    libharfbuzz.so.0 => /lib/x86_64-linux-gnu/libharfbuzz.so.0 (0x00007f15f3b60000)
    libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f15f3b20000)
    libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f15f3af1000)
    libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f15f3ae8000)
    libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f15f3aac000)
    libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007f15f3a9e000)
    libcairo-gobject.so.2 => /lib/x86_64-linux-gnu/libcairo-gobject.so.2 (0x00007f15f3a92000)
    libgdk_pixbuf-2.0.so.0 => /lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x00007f15f3a6c000)
    libgio-2.0.so.0 => /lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007f15f388e000)
    libpangocairo-1.0.so.0 => /lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 (0x00007f15f387b000)
    libpango-1.0.so.0 => /lib/x86_64-linux-gnu/libpango-1.0.so.0 (0x00007f15f3829000)
    libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007f15f381d000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f15f37aa000)
    libpixman-1.so.0 => /lib/x86_64-linux-gnu/libpixman-1.so.0 (0x00007f15f3704000)
    libxcb-shm.so.0 => /lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007f15f36fd000)
    libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f15f36d3000)
    libxcb-render.so.0 => /lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007f15f36c4000)
    libXrender.so.1 => /lib/x86_64-linux-gnu/libXrender.so.1 (0x00007f15f34ba000)
    libXext.so.6 => /lib/x86_64-linux-gnu/libXext.so.6 (0x00007f15f32a8000)
    libogg.so.0 => /lib/x86_64-linux-gnu/libogg.so.0 (0x00007f15f309f000)
    libnuma.so.1 => /lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f15f308f000)
    libXfixes.so.3 => /lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007f15f3087000)
    libicudata.so.67 => /lib/x86_64-linux-gnu/libicudata.so.67 (0x00007f15f156e000)
    libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f15f1494000)
    libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f15f1462000)
    libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f15f145c000)
    libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f15f144d000)
    libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007f15f1197000)
    libgraphite2.so.3 => /lib/x86_64-linux-gnu/libgraphite2.so.3 (0x00007f15f116b000)
    libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007f15f1146000)
    libgmodule-2.0.so.0 => /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007f15f1140000)
    libmount.so.1 => /lib/x86_64-linux-gnu/libmount.so.1 (0x00007f15f10e3000)
    libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f15f10b7000)
    libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f15f109d000)
    libpangoft2-1.0.so.0 => /lib/x86_64-linux-gnu/libpangoft2-1.0.so.0 (0x00007f15f1082000)
    libthai.so.0 => /lib/x86_64-linux-gnu/libthai.so.0 (0x00007f15f1077000)
    libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007f15f0e73000)
    libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f15f0c6d000)
    libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f15f0c66000)
    libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f15f0c1d000)
    libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007f15f0bca000)
    libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007f15f0b3a000)
    libdatrie.so.1 => /lib/x86_64-linux-gnu/libdatrie.so.1 (0x00007f15f0b30000)
frankier commented 3 years ago

Everyone experiencing this seems to be on Linux (Ubuntu or Debian). @trifle had libavdevice so that's probably not the problem. It is interesting that OS X seems to not have the problem. I wonder if the underlying problem is somehow related to this. If so, two places to consider are:

  1. Low level things like different kernel/system libraries somehow causing different behaviour (in which case this is probably an ffmpeg bug)
  2. Different flags passed when compiling ffmpeg, or Debian having applied a patch that changes this behaviour.

I think the next thing to do to track down what's going on here is for someone on Debian/Ubuntu to try compiling ffmpeg manually and see if the problem persists. If this fixes it the problem should be reported on the Debian bug tracker, otherwise the problem should be reported to ffmpeg.

ahkarami commented 3 years ago

Dear @innerlee & @zhreshold & @frankier & @trifle , 1- what is get_key_indices() do? 2- Would you please guide me what is the main difference between these methods (sequential read, random access(not accurate), and random access(accurate))? I think, the sequential read means:

vr = de.VideoReader(input_video)
for i in range(len(vr)):
    frame = vr.next()

and I think, the random access(not accurate) means:

vr = de.VideoReader(input_video)
for i in range(start_index, end_index):
    vr.seek(start_index)
    frame = vr.next()

and I think, the random access(accurate) means:

vr = de.VideoReader(input_video)
for i in range(start_index, end_index):    
    frame = vr[i]

or 

vr = de.VideoReader(input_video)
for i in range(start_index, end_index):    
    vr.seek_accurate(start_index)
    frame = vr.next()

am I correct? Best