search

benjiebob / SMALify

This repository contains an implementation for performing 3D animal (quadruped) reconstruction from a monocular image or video. The system adapts the pose (limb positions) and shape (animal type/height/weight) parameters for the SMAL deformable quadruped model, as well as camera parameters until the projected SMAL model aligns with 2D keypoints and silhouette segmentations extracted from the input frame(s).
110 stars 18 forks source link

File "/home/mona/research/3danimals/SMALify/smal_fitter/smal_fitter.py", line 198, in load_checkpoint with open(param_file, 'rb') as f: FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/20200930-121001/0000/st10_ep0.pkl' #8

Closed monajalal closed 4 years ago

monajalal commented 4 years ago 
(base) mona@mona:~/research/3danimals/SMALify$ python smal_fitter/generate_video.py

Bad key savefig.frameon in file /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle, line 421 ('savefig.frameon : True')
You probably need to get an updated matplotlibrc file from
https://github.com/matplotlib/matplotlib/blob/v3.3.2/matplotlibrc.template
or from the matplotlib source distribution

Bad key verbose.level in file /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle, line 472 ('verbose.level  : silent      # one of silent, helpful, debug, debug-annoying')
You probably need to get an updated matplotlibrc file from
https://github.com/matplotlib/matplotlib/blob/v3.3.2/matplotlibrc.template
or from the matplotlib source distribution

Bad key verbose.fileo in file /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle, line 473 ('verbose.fileo  : sys.stdout  # a log filename, sys.stdout or sys.stderr')
You probably need to get an updated matplotlibrc file from
https://github.com/matplotlib/matplotlib/blob/v3.3.2/matplotlibrc.template
or from the matplotlib source distribution
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The text.latex.preview rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The mathtext.fallback_to_cm rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: Support for setting the 'mathtext.fallback_to_cm' rcParam is deprecated since 3.3 and will be removed two minor releases later; use 'mathtext.fallback : 'cm' instead.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The validate_bool_maybe_none function was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The savefig.jpeg_quality rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The keymap.all_axes rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The animation.avconv_path rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The animation.avconv_args rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01<00:00,  9.36it/s]
Dataset size: 10
/home/mona/research/3danimals/SMALify/smal_fitter/priors/pose_prior_35.py:58: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1595629403081/work/torch/csrc/utils/tensor_numpy.cpp:141.)
  self.precs = torch.from_numpy(res['pic'].r).float().to(device)
Traceback (most recent call last):
  File "smal_fitter/generate_video.py", line 74, in <module>
    main()
  File "smal_fitter/generate_video.py", line 70, in main
    model.load_checkpoint(os.path.join("checkpoints", config.CHECKPOINT_NAME), config.EPOCH_NAME)
  File "/home/mona/research/3danimals/SMALify/smal_fitter/smal_fitter.py", line 198, in load_checkpoint
    with open(param_file, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/20200930-121001/0000/st10_ep0.pkl'

I have:

(base) mona@mona:~/research/3danimals/SMALify/checkpoints$ ls
total 40K
drwxrwxr-x 12 mona mona 4.0K Oct  1 01:37 20201001-013732
drwxrwxr-x 12 mona mona 4.0K Oct  1 01:57 20201001-015736
drwxrwxr-x 12 mona mona 4.0K Oct  1 02:22 20201001-022234
drwxrwxr-x 12 mona mona 4.0K Oct  1 02:30 20201001-023027
drwxrwxr-x 12 mona mona 4.0K Oct  1 02:32 20201001-023227
drwxrwxr-x 12 mona mona 4.0K Oct  1 02:33 20201001-023328
drwxrwxr-x 12 mona mona 4.0K Oct  1 02:35 20201001-023459
drwxrwxr-x 10 mona mona 4.0K Oct  1 02:50 .
drwxrwxr-x 12 mona mona 4.0K Oct  1 02:50 20201001-025010
drwxrwxr-x 11 mona mona 4.0K Oct  1 03:04 ..

I am using this version of the code:

(base) mona@mona:~/research/3danimals/SMALify/checkpoints$ git log -1
commit 01b56564a62b3b85df64a3168755c9878865e0bf (HEAD -> master, origin/master, origin/HEAD)
Author: Benjamin Biggs <benjbiggs@outlook.com>
Date:   Wed Sep 30 18:23:51 2020 +0100

    Update README.md
benjiebob commented 4 years ago

This error has occured because the CHECKPOINT_NAME in config.py is still set to a default "20200930-121001" (which is an output directory on my machine). You should update this to the name of your own output directory, generated by the optimize_to_joints.py script.

monajalal commented 4 years ago

Thanks a lot for feedback.

changed my config.py file to:

# Generate video settings
CHECKPOINT_NAME = "20201001-025010" # the directory to run
EPOCH_NAME = "st10_ep0" # convention used for the final output file. Don't change this without good reason.

and reran:


(base) mona@mona:~/research/3danimals/SMALify$ python smal_fitter/generate_video.py

Bad key savefig.frameon in file /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle, line 421 ('savefig.frameon : True')
You probably need to get an updated matplotlibrc file from
https://github.com/matplotlib/matplotlib/blob/v3.3.2/matplotlibrc.template
or from the matplotlib source distribution

Bad key verbose.level in file /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle, line 472 ('verbose.level  : silent      # one of silent, helpful, debug, debug-annoying')
You probably need to get an updated matplotlibrc file from
https://github.com/matplotlib/matplotlib/blob/v3.3.2/matplotlibrc.template
or from the matplotlib source distribution

Bad key verbose.fileo in file /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle, line 473 ('verbose.fileo  : sys.stdout  # a log filename, sys.stdout or sys.stderr')
You probably need to get an updated matplotlibrc file from
https://github.com/matplotlib/matplotlib/blob/v3.3.2/matplotlibrc.template
or from the matplotlib source distribution
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The text.latex.preview rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The mathtext.fallback_to_cm rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: Support for setting the 'mathtext.fallback_to_cm' rcParam is deprecated since 3.3 and will be removed two minor releases later; use 'mathtext.fallback : 'cm' instead.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The validate_bool_maybe_none function was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The savefig.jpeg_quality rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The keymap.all_axes rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The animation.avconv_path rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
In /home/mona/anaconda3/lib/python3.7/site-packages/matplotlib/mpl-data/stylelib/_classic_test.mplstyle: 
The animation.avconv_args rcparam was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01<00:00,  6.36it/s]
Dataset size: 10
/home/mona/research/3danimals/SMALify/smal_fitter/priors/pose_prior_35.py:58: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /opt/conda/conda-bld/pytorch_1595629403081/work/torch/csrc/utils/tensor_numpy.cpp:141.)
  self.precs = torch.from_numpy(res['pic'].r).float().to(device)

(base) mona@mona:~/research/3danimals/SMALify$ ls exported/
total 20K
drwxrwxr-x  3 mona mona 4.0K Oct  1 03:04 20200930-121001
drwxrwxr-x  3 mona mona 4.0K Oct  1 23:40 20201001-125009
drwxrwxr-x 11 mona mona 4.0K Oct  1 23:45 ..
drwxrwxr-x  3 mona mona 4.0K Oct  1 23:45 20201001-025010
drwxrwxr-x  5 mona mona 4.0K Oct  1 23:45 .

(base) mona@mona:~/research/3danimals/SMALify/exported/20201001-025010/st10_ep0$ ffmpeg -framerate 2 -pattern_type glob -i '*.png' -pix_fmt yuv420p results.mp4
ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  WARNING: library configuration mismatch
  avcodec     configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libaribb24 --enable-liblensfun --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, image2, from '*.png':
  Duration: 00:00:05.00, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: png, rgb24(pc), 1280x256, 2 fps, 2 tbr, 2 tbn, 2 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x5652f00e2480] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x5652f00e2480] profile High, level 2.2
[libx264 @ 0x5652f00e2480] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=8 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=2 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'results.mp4':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x256, q=-1--1, 2 fps, 16384 tbn, 2 tbc
    Metadata:
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame=   10 fps=0.0 q=-1.0 Lsize=     194kB time=00:00:03.50 bitrate= 453.0kbits/s speed=21.8x    
video:193kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.480579%
[libx264 @ 0x5652f00e2480] frame I:1     Avg QP:12.59  size: 37536
[libx264 @ 0x5652f00e2480] frame P:3     Avg QP:16.04  size: 23302
[libx264 @ 0x5652f00e2480] frame B:6     Avg QP:18.00  size: 14856
[libx264 @ 0x5652f00e2480] consecutive B-frames: 10.0% 20.0% 30.0% 40.0%
[libx264 @ 0x5652f00e2480] mb I  I16..4: 34.1% 28.0% 37.8%
[libx264 @ 0x5652f00e2480] mb P  I16..4:  4.7% 10.9% 12.0%  P16..4: 13.0% 15.5%  9.1%  0.0%  0.0%    skip:34.8%
[libx264 @ 0x5652f00e2480] mb B  I16..4:  0.4%  0.5%  1.4%  B16..8: 21.5% 20.4% 11.7%  direct: 3.7%  skip:40.3%  L0:48.1% L1:36.6% BI:15.3%
[libx264 @ 0x5652f00e2480] 8x8 transform intra:32.5% inter:43.5%
[libx264 @ 0x5652f00e2480] coded y,uvDC,uvAC intra: 57.1% 47.0% 37.6% inter: 26.2% 24.1% 15.2%
[libx264 @ 0x5652f00e2480] i16 v,h,dc,p: 78% 14%  6%  2%
[libx264 @ 0x5652f00e2480] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  8% 49% 24%  2%  3%  2%  5%  2%  6%
[libx264 @ 0x5652f00e2480] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 27% 20%  5%  6%  6%  6%  5%  6%
[libx264 @ 0x5652f00e2480] i8c dc,h,v,p: 62% 22% 13%  3%
[libx264 @ 0x5652f00e2480] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x5652f00e2480] ref P L0: 74.1%  7.8% 11.0%  7.2%
[libx264 @ 0x5652f00e2480] ref B L0: 91.7%  7.5%  0.8%
[libx264 @ 0x5652f00e2480] ref B L1: 98.5%  1.5%
[libx264 @ 0x5652f00e2480] kb/s:314.52

(base) mona@mona:~/research/3danimals/SMALify/exported/20201001-025010/st10_ep0$ ls
total 2.4M
drwxrwxr-x 3 mona mona 4.0K Oct  1 23:45 ..
-rw-rw-r-- 1 mona mona 186K Oct  1 23:45 0000.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0000.pkl
-rw-rw-r-- 1 mona mona 183K Oct  1 23:45 0001.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0001.pkl
-rw-rw-r-- 1 mona mona 179K Oct  1 23:45 0002.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0002.pkl
-rw-rw-r-- 1 mona mona 146K Oct  1 23:45 0003.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0003.pkl
-rw-rw-r-- 1 mona mona 152K Oct  1 23:45 0004.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0004.pkl
-rw-rw-r-- 1 mona mona 152K Oct  1 23:45 0005.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0005.pkl
-rw-rw-r-- 1 mona mona 154K Oct  1 23:45 0006.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0006.pkl
-rw-rw-r-- 1 mona mona 154K Oct  1 23:45 0007.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0007.pkl
-rw-rw-r-- 1 mona mona 155K Oct  1 23:45 0008.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0008.pkl
-rw-rw-r-- 1 mona mona 158K Oct  1 23:45 0009.png
-rw-rw-r-- 1 mona mona  933 Oct  1 23:45 0009.pkl
-rw-rw-r-- 1 mona mona 194K Oct  1 23:47 results.mp4
drwxrwxr-x 2 mona mona 4.0K Oct  1 23:49 .

(base) mona@mona:~/research/3danimals/SMALify/exported/20201001-025010/st10_ep0$ ffmpeg -i results.mp4 res.gif

res