search

talmolab / sleap

A deep learning framework for multi-animal pose tracking.
https://sleap.ai
Other
435 stars 97 forks source link

Still receive `KeyError` even after re-encoding #1707

Closed rhansen123 closed 8 months ago

rhansen123 commented 8 months ago

Bug description

I have been trying to run batches of videos through SLEAP inference, but SLEAP has had issues with a few videos. The error in the command terminal is attached to this post, but it seems to have an issue with a specific frame in a video and not finish running an inference. I reencoded the videos through Python as suggested, but the issue still occurs.

Expected behaviour

SLEAP should be able to run inference on the videos.

Actual behaviour

It stops at a few different videos with an error that it cannot load a frame. Capture

Environment packages ``` # paste output of `pip freeze` or `conda list` here ```
Logs ``` # paste relevant logs here, if any ```

Screenshots

How to reproduce

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error
roomrys commented 8 months ago

Hey @rhansen123,

Glad that you tried the re-encoding. That usually does the trick. Not glad it didn't work. The useful piece of information here would be what happened after you tried re-encoding your video? Did the terminal say anything? I expect it should have since it sounds like the re-encoding failed and that's why things are not working.

Thanks, Liezl

rhansen123 commented 8 months ago

Hello Liezl,

The videos were successfully reencoded, but had the same issue with SLEAP not being able to process through the video. I took a screenshot of the command prompt and attached it to this email, but I also copied the command terminal and pasted it below this email. There is no error and the resulting video loads like normal. I am not sure what else to do?

Best, Rachael

C:\Users\Nape_Computer_1>ffmpeg -y -i
"Z:\Golden_Lab_Users\Hansen_Rachael\ISOTRAP012024_Analysis\Done_Top\5mgkg_593_cno2_Box2Top-20240122T103722-110910.avi"
-c:v libx264 -pix_fmt yuv420p -preset superfast -crf 23
"Z:\Golden_Lab_Users\Hansen_Rachael\ISOTRAP012024_Analysis\5mgkg_593_cno2_Box2Top-20240122T103722-110910_reencoded.avi"
ffmpeg version 2021-10-07-git-b6aeee2d8b-full_build-www.gyan.dev Copyright
(c) 2000-2021 the FFmpeg developers
  built with gcc 10.3.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static
--disable-w32threads --disable-autodetect --enable-fontconfig
--enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma
--enable-libsnappy --enable-zlib --enable-librist --enable-libsrt
--enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray
--enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi
--enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264
--enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg
--enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype
--enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg
--enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec
--enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2
--enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl
--enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt
--enable-libopencore-amrwb --enable-libmp3lame --enable-libshine
--enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc
--enable-libilbc --enable-libgsm --enable-libopencore-amrnb
--enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa
--enable-libbs2b --enable-libflite --enable-libmysofa
--enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59. 10.100 / 59. 10.100
  libavformat    59.  5.101 / 59.  5.101
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 11.100 /  8. 11.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, avi, from
'Z:\Golden_Lab_Users\Hansen_Rachael\ISOTRAP012024_Analysis\Done_Top\5mgkg_593_cno2_Box2Top-20240122T103722-110910.avi':
  Metadata:
    software        : Lavf59.5.101
  Duration: 00:30:00.03, start: 0.000000, bitrate: 1959 kb/s
  Stream #0:0: Video: h264 (Main) (H264 / 0x34363248), yuvj420p(pc,
bt470bg/unknown/unknown, progressive), 860x716 [SAR 1:1 DAR 215:179], 1953
kb/s, 30 fps, 30 tbr, 30 tbn
    Metadata:
      title           : Created with https://github.com/icza/mjpeg at
2024-01-22 10:37:22 PST
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174beec00] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174c14000] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174c40c40] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174c6eb00] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174ca3fc0] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174ccefc0] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174d08e40] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174d57f80] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174d6efc0] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174d9dd40] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174daf100] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174dd8f40] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174df5f40] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174e48ec0] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174e71f40] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174ea1e40] deprecated
pixel format used, make sure you did set range correctly
[swscaler @ 000001b174bdd800] [swscaler @ 000001b174eaee00] deprecated
pixel format used, make sure you did set range correctly
[libx264 @ 000001b171f666c0] using SAR=1/1
[libx264 @ 000001b171f666c0] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001b171f666c0] profile High, level 3.1, 4:2:0, 8-bit
Output #0, avi, to
'Z:\Golden_Lab_Users\Hansen_Rachael\ISOTRAP012024_Analysis\5mgkg_593_cno2_Box2Top-20240122T103722-110910_reencoded.avi':
  Metadata:
    software        : Lavf59.5.101
    ISFT            : Lavf59.5.101
  Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(tv,
bt470bg/unknown/unknown, progressive), 860x716 [SAR 1:1 DAR 215:179],
q=2-31, 30 fps, 30 tbn
    Metadata:
      title           : Created with https://github.com/icza/mjpeg at
2024-01-22 10:37:22 PST
      encoder         : Lavc59.10.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=52920 fps=1013 q=-1.0 Lsize=  113297kB time=00:30:00.00 bitrate=
515.6kbits/s dup=0 drop=5 speed=34.5x
video:112000kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 1.158169%
[libx264 @ 000001b171f666c0] frame I:212   Avg QP:13.62  size: 45076
[libx264 @ 000001b171f666c0] frame P:13347 Avg QP:14.48  size:  5042
[libx264 @ 000001b171f666c0] frame B:39361 Avg QP:15.40  size:   961
[libx264 @ 000001b171f666c0] consecutive B-frames:  0.6%  0.4%  0.6% 98.4%
[libx264 @ 000001b171f666c0] mb I  I16..4:  5.0% 32.6% 62.5%
[libx264 @ 000001b171f666c0] mb P  I16..4:  1.3%  3.0%  0.7%  P16..4: 38.0%
 0.0%  0.0%  0.0%  0.0%    skip:57.0%
[libx264 @ 000001b171f666c0] mb B  I16..4:  0.3%  0.2%  0.0%  B16..8: 10.0%
 0.0%  0.0%  direct: 3.4%  skip:86.0%  L0:39.1% L1:55.9% BI: 5.0%
[libx264 @ 000001b171f666c0] 8x8 transform intra:51.0% inter:23.8%
[libx264 @ 000001b171f666c0] coded y,uvDC,uvAC intra: 60.3% 0.3% 0.1%
inter: 4.9% 0.0% 0.0%
[libx264 @ 000001b171f666c0] i16 v,h,dc,p: 39% 28% 25%  8%
[libx264 @ 000001b171f666c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 21% 28%  5%
 6%  6%  5%  5%  6%
[libx264 @ 000001b171f666c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 22% 15%  9%
 7%  5%  6%  5%  8%
[libx264 @ 000001b171f666c0] i8c dc,h,v,p: 99%  1%  0%  0%
[libx264 @ 000001b171f666c0] Weighted P-Frames: Y:0.1% UV:0.0%
[libx264 @ 000001b171f666c0] kb/s:509.72
talmo commented 8 months ago

Closing this in favor of #1711.

@rhansen123: This will be fixed in the next release, but feel free to install from source with the latest version from the develop branch if you're blocked.

roaldarbol commented 4 months ago

@talmo Came across this issue as I was running into the same issue (I resolved it in the end, but wasn't that easy to sort out). When do you expect the next release to come out? :-)

talmo commented 4 months ago

Soon!™ @eberrigan @roomrys

roaldarbol commented 4 months ago

Haha, I'll take as a "let's hear what the other folks think" 😂 Looking forward to it nevertheless!