search

xg-chu / lightning_track

[ICLR 2024] Generalizable and Precise Head Avatar from Image(s)
https://xg-chu.site/project_gpavatar/
60 stars 7 forks source link

IndexError: list index out of range #1

Closed z89685413 closed 7 months ago

z89685413 commented 7 months ago

When executing track_video.py with the python track_video.py -v demos/demo.mp4 --synthesis option for video processing, I encountered an IndexError during the 'Track lightning' phase.

Processing video data... Track with emoca/mica/insightface/mediapipe/... Track lightning... Traceback (most recent call last): File "/home/yyds/xnr/GPAvatar/lightning_track/track_video.py", line 231, in tracker.track_video(args.video_path, dir_path=args.outdir_path, synthesis=args.synthesis, no_vis=args.no_vis) File "/home/yyds/xnr/GPAvatar/lightning_track/track_video.py", line 32, in track_video lightning_results = self.run_lightning(base_results, lmdb_engine, output_path) File "/home/yyds/xnr/GPAvatar/lightning_track/track_video.py", line 60, in run_lightning lightning_result = self.tracker.track_lightning(base_results, lmdb_engine, output_path) File "/home/yyds/xnr/GPAvatar/lightning_track/engines/core_engine.py", line 82, in track_lightning batch_frames = torch.stack([lmdb_engine[key] for key in mini_batchs[0]][:20]).to(self._device).float() IndexError: list index out of range /home/yyds/xnr/GPAvatar/lightning_track/utils/lmdb_utils.py:46: RuntimeWarning: Writing engine not mannuly closed! warn('Writing engine not mannuly closed!', RuntimeWarning)

xg-chu commented 7 months ago

For the demo video it looks really strange, could you please give me the environment list (conda/pip)? Maybe some dependencies are different. Also, can you check the file size of the generate lmdb folder and base.pkl? I want to know if they are empty.

z89685413 commented 7 months ago

Package Version

absl-py 2.0.0 attrs 23.1.0 av 11.0.0 Brotli 1.0.9 certifi 2023.11.17 cffi 1.16.0 charset-normalizer 2.0.4 colorama 0.4.6 colored 2.2.3 coloredlogs 15.0.1 contourpy 1.2.0 cryptography 41.0.3 cycler 0.12.1 einops 0.7.0 filelock 3.13.1 flatbuffers 23.5.26 fonttools 4.46.0 fsspec 2023.12.1 fvcore 0.1.5.post20221221 gmpy2 2.1.2 huggingface-hub 0.19.4 humanfriendly 10.0 idna 3.4 imageio 2.33.0 importlib-resources 6.1.1 iopath 0.1.9 Jinja2 3.1.2 kiwisolver 1.4.5 lazy_loader 0.3 lmdb 1.4.1 markdown-it-py 3.0.0 MarkupSafe 2.1.1 matplotlib 3.8.2 mdurl 0.1.2 mediapipe 0.10.7 mkl-fft 1.3.8 mkl-random 1.2.4 mkl-service 2.4.0 mpmath 1.3.0 networkx 3.1 ninja 1.11.1.1 numpy 1.26.2 onnx 1.15.0 onnxruntime-gpu 1.16.3 opencv-contrib-python 4.8.1.78 opencv-python 4.8.1.78 packaging 23.2 Pillow 10.0.1 pip 23.3.1 portalocker 2.8.2 protobuf 3.20.3 pycparser 2.21 Pygments 2.17.2 pykalman 0.9.5 pyOpenSSL 23.2.0 pyparsing 3.1.1 PySocks 1.7.1 python-dateutil 2.8.2 pytorch3d 0.7.5 PyYAML 6.0 regex 2023.10.3 requests 2.31.0 rich 13.7.0 safetensors 0.4.1 scikit-image 0.22.0 scipy 1.11.4 setuptools 68.0.0 six 1.16.0 sounddevice 0.4.6 sympy 1.12 tabulate 0.9.0 termcolor 2.3.0 tifffile 2023.9.26 tokenizers 0.15.0 torch 2.0.1 torchaudio 2.0.2 torchvision 0.15.2 tqdm 4.66.1 transformers 4.35.2 triton 2.0.0 typing_extensions 4.7.1 urllib3 1.26.18 wheel 0.41.2 yacs 0.1.8 zipp 3.17.0

xg-chu commented 7 months ago

Could you also give me the results of "conda list" and the file size of the generate lmdb folder and base.pkl?

z89685413 commented 7 months ago

yeah, sure

Name Version Build Channel

_libgcc_mutex 0.1 main conda-forge _openmp_mutex 5.1 1_gnu defaults absl-py 2.0.0 pypi_0 pypi attrs 23.1.0 pypi_0 pypi av 11.0.0 pypi_0 pypi blas 1.0 mkl conda-forge brotli-python 1.0.9 py39h6a678d5_7 defaults bzip2 1.0.8 h7b6447c_0 defaults ca-certificates 2023.11.17 hbcca054_0 conda-forge certifi 2023.11.17 pyhd8ed1ab_0 conda-forge cffi 1.16.0 py39h5eee18b_0 defaults charset-normalizer 2.0.4 pyhd3eb1b0_0 defaults colorama 0.4.6 pyhd8ed1ab_0 conda-forge colored 2.2.3 pypi_0 pypi coloredlogs 15.0.1 pypi_0 pypi contourpy 1.2.0 pypi_0 pypi cryptography 41.0.3 py39hdda0065_0 defaults cuda-cudart 11.8.89 0 nvidia cuda-cupti 11.8.87 0 nvidia cuda-libraries 11.8.0 0 nvidia cuda-nvrtc 11.8.89 0 nvidia cuda-nvtx 11.8.86 0 nvidia cuda-runtime 11.8.0 0 nvidia cycler 0.12.1 pypi_0 pypi einops 0.7.0 pypi_0 pypi ffmpeg 4.3 hf484d3e_0 pytorch filelock 3.13.1 py39h06a4308_0 defaults flatbuffers 23.5.26 pypi_0 pypi fonttools 4.46.0 pypi_0 pypi freetype 2.12.1 h4a9f257_0 defaults fsspec 2023.12.1 pypi_0 pypi fvcore 0.1.5.post20221221 pyhd8ed1ab_0 conda-forge giflib 5.2.1 h5eee18b_3 defaults gmp 6.2.1 h295c915_3 defaults gmpy2 2.1.2 py39heeb90bb_0 defaults gnutls 3.6.15 he1e5248_0 defaults huggingface-hub 0.19.4 pypi_0 pypi humanfriendly 10.0 pypi_0 pypi idna 3.4 py39h06a4308_0 defaults imageio 2.33.0 pypi_0 pypi importlib-resources 6.1.1 pypi_0 pypi intel-openmp 2023.1.0 hdb19cb5_46306 defaults iopath 0.1.9 py39 iopath jinja2 3.1.2 py39h06a4308_0 defaults jpeg 9e h5eee18b_1 defaults kiwisolver 1.4.5 pypi_0 pypi lame 3.100 h7b6447c_0 defaults lazy-loader 0.3 pypi_0 pypi lcms2 2.12 h3be6417_0 defaults ld_impl_linux-64 2.38 h1181459_1 defaults lerc 3.0 h295c915_0 defaults libcublas 11.11.3.6 0 nvidia libcufft 10.9.0.58 0 nvidia libcufile 1.8.1.2 0 nvidia libcurand 10.3.4.101 0 nvidia libcusolver 11.4.1.48 0 nvidia libcusparse 11.7.5.86 0 nvidia libdeflate 1.17 h5eee18b_1 defaults libffi 3.4.4 h6a678d5_0 defaults libgcc-ng 11.2.0 h1234567_1 defaults libgomp 11.2.0 h1234567_1 defaults libiconv 1.16 h7f8727e_2 defaults libidn2 2.3.4 h5eee18b_0 defaults libnpp 11.8.0.86 0 nvidia libnvjpeg 11.9.0.86 0 nvidia libpng 1.6.39 h5eee18b_0 defaults libstdcxx-ng 11.2.0 h1234567_1 defaults libtasn1 4.19.0 h5eee18b_0 defaults libtiff 4.5.1 h6a678d5_0 defaults libunistring 0.9.10 h27cfd23_0 defaults libwebp 1.3.2 h11a3e52_0 defaults libwebp-base 1.3.2 h5eee18b_0 defaults lmdb 1.4.1 pypi_0 pypi lz4-c 1.9.4 h6a678d5_0 defaults markdown-it-py 3.0.0 pypi_0 pypi markupsafe 2.1.1 py39h7f8727e_0 defaults matplotlib 3.8.2 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi mediapipe 0.10.7 pypi_0 pypi mkl 2023.1.0 h213fc3f_46344 defaults mkl-service 2.4.0 py39h5eee18b_1 defaults mkl_fft 1.3.8 py39h5eee18b_0 defaults mkl_random 1.2.4 py39hdb19cb5_0 defaults mpc 1.1.0 h10f8cd9_1 defaults mpfr 4.0.2 hb69a4c5_1 defaults mpmath 1.3.0 py39h06a4308_0 defaults ncurses 6.4 h6a678d5_0 defaults nettle 3.7.3 hbbd107a_1 defaults networkx 3.1 py39h06a4308_0 defaults ninja 1.11.1.1 pypi_0 pypi numpy 1.26.2 py39h5f9d8c6_0 defaults numpy-base 1.26.2 py39hb5e798b_0 defaults onnx 1.15.0 pypi_0 pypi onnxruntime-gpu 1.16.3 pypi_0 pypi opencv-contrib-python 4.8.1.78 pypi_0 pypi opencv-python 4.8.1.78 pypi_0 pypi openh264 2.1.1 h4ff587b_0 defaults openjpeg 2.4.0 h3ad879b_0 defaults openssl 3.0.12 h7f8727e_0 defaults packaging 23.2 pypi_0 pypi pillow 10.0.1 py39ha6cbd5a_0 defaults pip 23.3.1 py39h06a4308_0 defaults portalocker 2.8.2 py39hf3d152e_1 conda-forge protobuf 3.20.3 pypi_0 pypi pycparser 2.21 pyhd3eb1b0_0 defaults pygments 2.17.2 pypi_0 pypi pykalman 0.9.5 pypi_0 pypi pyopenssl 23.2.0 py39h06a4308_0 defaults pyparsing 3.1.1 pypi_0 pypi pysocks 1.7.1 py39h06a4308_0 defaults python 3.9.18 h955ad1f_0 defaults python-dateutil 2.8.2 pypi_0 pypi python_abi 3.9 2_cp39 conda-forge pytorch 2.0.1 py3.9_cuda11.8_cudnn8.7.0_0 pytorch pytorch-cuda 11.8 h7e8668a_5 pytorch pytorch-mutex 1.0 cuda pytorch pytorch3d 0.7.5 py39_cu118_pyt201 pytorch3d pyyaml 6.0 py39hb9d737c_4 conda-forge readline 8.2 h5eee18b_0 defaults regex 2023.10.3 pypi_0 pypi requests 2.31.0 py39h06a4308_0 defaults rich 13.7.0 pypi_0 pypi safetensors 0.4.1 pypi_0 pypi scikit-image 0.22.0 pypi_0 pypi scipy 1.11.4 pypi_0 pypi setuptools 68.0.0 py39h06a4308_0 defaults six 1.16.0 pypi_0 pypi sounddevice 0.4.6 pypi_0 pypi sqlite 3.41.2 h5eee18b_0 defaults sympy 1.12 py39h06a4308_0 defaults tabulate 0.9.0 pyhd8ed1ab_1 conda-forge tbb 2021.8.0 hdb19cb5_0 defaults termcolor 2.3.0 pyhd8ed1ab_0 conda-forge tifffile 2023.9.26 pypi_0 pypi tk 8.6.12 h1ccaba5_0 defaults tokenizers 0.15.0 pypi_0 pypi torchaudio 2.0.2 py39_cu118 pytorch torchtriton 2.0.0 py39 pytorch torchvision 0.15.2 py39_cu118 pytorch tqdm 4.66.1 pyhd8ed1ab_0 conda-forge transformers 4.35.2 pypi_0 pypi typing_extensions 4.7.1 py39h06a4308_0 defaults tzdata 2023c h04d1e81_0 defaults urllib3 1.26.18 py39h06a4308_0 defaults wheel 0.41.2 py39h06a4308_0 defaults xz 5.4.5 h5eee18b_0 defaults yacs 0.1.8 pyhd8ed1ab_0 conda-forge yaml 0.2.5 h7f98852_2 conda-forge zipp 3.17.0 pypi_0 pypi zlib 1.2.13 h5eee18b_0 defaults zstd 1.5.5 hc292b87_0 defaults

z89685413 commented 7 months ago

file size of the generate lmdb folder and base.pkl appreciated for your help, where are those files located?

xg-chu commented 7 months ago

file size of the generate lmdb folder and base.pkl appreciated for your help, where are those files located?

They should be in lightning_track/outputs/demo/[img_lmdb/base.pkl]

z89685413 commented 7 months ago

[img_lmdb/ --data.mdb 8.2KB --lock.mdb 8.2KB base.pkl/ 5 byte

xg-chu commented 7 months ago

[img_lmdb/ --data.mdb 8.2KB --lock.mdb 8.2KB base.pkl/ 5 byte

They are empty, the problem maybe easier to solve... Could you give me the output of "ffmpeg -version"? I will try to find and fix the problem now.

z89685413 commented 7 months ago

ffmpeg version 4.3 Copyright (c) 2000-2020 the FFmpeg developers built with gcc 7.3.0 (crosstool-NG 1.23.0.449-a04d0) configuration: --prefix=/opt/conda/conda-bld/ffmpeg_1597178665428/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh --cc=/opt/conda/conda-bld/ffmpeg_1597178665428/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc --disable-doc --disable-openssl --enable-avresample --enable-gnutls --enable-hardcoded-tables --enable-libfreetype --enable-libopenh264 --enable-pic --enable-pthreads --enable-shared --disable-static --enable-version3 --enable-zlib --enable-libmp3lame libavutil 56. 51.100 / 56. 51.100 libavcodec 58. 91.100 / 58. 91.100 libavformat 58. 45.100 / 58. 45.100 libavdevice 58. 10.100 / 58. 10.100 libavfilter 7. 85.100 / 7. 85.100 libavresample 4. 0. 0 / 4. 0. 0 libswscale 5. 7.100 / 5. 7.100 libswresample 3. 7.100 / 3. 7.100

xg-chu commented 7 months ago

Upgrade the codebase and try again? I tried to bypass some beta features of torchvision and adding some assertion to help diagnose.

z89685413 commented 7 months ago

Upgrade the codebase and try again? I tried to bypass some beta features of torchvision and adding some assertion to help diagnose.

thanks i just tried but find the output directory is gone, am i missing something?

xg-chu commented 7 months ago

Upgrade the codebase and try again? I tried to bypass some beta features of torchvision and adding some assertion to help diagnose.

thanks i just tried but find the output directory is gone, am i missing something?

Sorry I made some mistakes, it is fixed now.

StarSapph1re commented 7 months ago

I have run into the same problem. I deleted the output directory generated when I run track_video.py before, for example when I try to track a video "1.mp4" I should ensure that there's no directory named 1 in "./output". Hope this work for you.

z89685413 commented 7 months ago

Upgrade the codebase and try again? I tried to bypass some beta features of torchvision and adding some assertion to help diagnose.

thanks i just tried but find the output directory is gone, am i missing something?

Sorry I made some mistakes, it is fixed now.

thanks

xg-chu commented 7 months ago

I have run into the same problem. I deleted the output directory generated when I run track_video.py before, for example when I try to track a video "1.mp4" I should ensure that there's no directory named 1 in "./output". Hope this work for you.

That's it, now it automatically removes the old directory in "./outputs"🥹