search

zju3dv / 4K4D

[CVPR 2024] 4K4D: Real-Time 4D View Synthesis at 4K Resolution
https://zju3dv.github.io/4k4d/
Other
1.6k stars 69 forks source link

error in imgui_bundle #28

Open hhhddddddd opened 8 months ago

hhhddddddd commented 8 months ago

Hello, your work is very interesting！ I execute evc -t gui -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/video.yaml on the Minimal Dataset. I have the following error: problem0 problem1 then, I followed the prompts to install imgui_bundle. however, I have a new problem:

`(4k4d) xuankai@com4-X780-G30:~/code/4K4D$ pip install imgui_bundle Collecting imgui_bundle Using cached imgui-bundle-1.3.0.tar.gz (36.8 MB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: numpy>=1.15 in /home/xuankai/anaconda3/envs/4k4d/lib/python3.9/site-packages (from imgui_bundle) (1.26.4) Collecting munch>=2.0.0 (from imgui_bundle) Using cached munch-4.0.0-py2.py3-none-any.whl.metadata (5.9 kB) Collecting glfw>2.5 (from imgui_bundle) Using cached glfw-2.7.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-manylinux2014_x86_64.whl.metadata (5.4 kB) Collecting PyOpenGL>=3.0 (from imgui_bundle) Using cached PyOpenGL-3.1.7-py3-none-any.whl.metadata (3.2 kB) Requirement already satisfied: pillow>=9.0.0 in /home/xuankai/anaconda3/envs/4k4d/lib/python3.9/site-packages (from imgui_bundle) (10.2.0) Using cached glfw-2.7.0-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38-none-manylinux2014_x86_64.whl (211 kB) Using cached munch-4.0.0-py2.py3-none-any.whl (9.9 kB) Using cached PyOpenGL-3.1.7-py3-none-any.whl (2.4 MB) Building wheels for collected packages: imgui_bundle Building wheel for imgui_bundle (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for imgui_bundle (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [118 lines of output]

  --------------------------------------------------------------------------------
  -- Trying 'Ninja' generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
    Compatibility with CMake < 3.5 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value or use a ...<max> suffix to tell
    CMake that the project does not need compatibility with older versions.

  Not searching for unused variables given on the command line.

  -- The C compiler identification is GNU 9.4.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is GNU 9.4.0
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done (2.6s)
  -- Generating done (0.0s)
  -- Build files have been written to: /tmp/pip-install-10dc4lcj/imgui-bundle_c98c9f82c8004e88b0d20ceb0adac3a6/_cmake_test_compile/build
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying 'Ninja' generator - success
  --------------------------------------------------------------------------------

  Configuring Project
    Working directory:
      /tmp/pip-install-10dc4lcj/imgui-bundle_c98c9f82c8004e88b0d20ceb0adac3a6/_skbuild/linux-x86_64-3.9/cmake-build
    Command:
      /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/cmake/data/bin/cmake /tmp/pip-install-10dc4lcj/imgui-bundle_c98c9f82c8004e88b0d20ceb0adac3a6 -G Ninja -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-install-10dc4lcj/imgui-bundle_c98c9f82c8004e88b0d20ceb0adac3a6/_skbuild/linux-x86_64-3.9/cmake-install/bindings/imgui_bundle -DPYTHON_VERSION_STRING:STRING=3.9.18 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/home/xuankai/anaconda3/envs/4k4d/bin/python -DPYTHON_INCLUDE_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d/include/python3.9 -DPYTHON_LIBRARY:PATH=/home/xuankai/anaconda3/envs/4k4d/lib/libpython3.9.so -DPython_EXECUTABLE:PATH=/home/xuankai/anaconda3/envs/4k4d/bin/python -DPython_ROOT_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d/include/python3.9 -DPython3_EXECUTABLE:PATH=/home/xuankai/anaconda3/envs/4k4d/bin/python -DPython3_ROOT_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d/include/python3.9 -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DCMAKE_BUILD_TYPE:STRING=Release

  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 9.4.0
  -- The CXX compiler identification is GNU 9.4.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- _him_check_if_no_backend_selected return ON
  -- Cleaning bindings/imgui_bundle before pip build
  CMake Warning (dev) at /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/FindPythonLibsNew.cmake:101 (message):
    Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
    are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
    the cmake_policy command to set the policy and suppress this warning, or
    preferably upgrade to using FindPython, either by calling it explicitly
    before pybind11, or by setting PYBIND11_FINDPYTHON ON before pybind11.
  Call Stack (most recent call first):
    /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Tools.cmake:50 (find_package)
    /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Common.cmake:192 (include)
    /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/pybind11/share/cmake/pybind11/pybind11Config.cmake:250 (include)
    cmake/find_pybind11.cmake:20 (find_package)
    CMakeLists.txt:329 (find_pybind11)
  This warning is for project developers.  Use -Wno-dev to suppress it.
  '/home/xuankai/anaconda3/envs/4k4d/bin/python' '-c' 'import pybind11; print(pybind11.get_cmake_dir())'

  -- Found PythonInterp: /home/xuankai/anaconda3/envs/4k4d/bin/python (found suitable version "3.9.18", minimum required is "3.6")
  -- Found PythonLibs: /home/xuankai/anaconda3/envs/4k4d/lib/libpython3.9.so
  -- Performing Test HAS_FLTO
  -- Performing Test HAS_FLTO - Success
  -- Found pybind11: /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/pybind11/include (found version "2.12.0")
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
  -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE
  -- Using X11 for window creation
  CMake Error at /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/cmake/data/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
    Could NOT find X11 (missing: X11_X11_INCLUDE_PATH X11_X11_LIB)
  Call Stack (most recent call first):
    /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/cmake/data/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
    /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/cmake/data/share/cmake-3.29/Modules/FindX11.cmake:676 (find_package_handle_standard_args)
    external/glfw/glfw/CMakeLists.txt:208 (find_package)

  -- Configuring incomplete, errors occurred!
  Traceback (most recent call last):
    File "/tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/skbuild/setuptools_wrap.py", line 666, in setup
      env = cmkr.configure(
    File "/tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/skbuild/cmaker.py", line 357, in configure
      raise SKBuildError(msg)

  An error occurred while configuring with CMake.
    Command:
      /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/cmake/data/bin/cmake /tmp/pip-install-10dc4lcj/imgui-bundle_c98c9f82c8004e88b0d20ceb0adac3a6 -G Ninja -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-install-10dc4lcj/imgui-bundle_c98c9f82c8004e88b0d20ceb0adac3a6/_skbuild/linux-x86_64-3.9/cmake-install/bindings/imgui_bundle -DPYTHON_VERSION_STRING:STRING=3.9.18 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/home/xuankai/anaconda3/envs/4k4d/bin/python -DPYTHON_INCLUDE_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d/include/python3.9 -DPYTHON_LIBRARY:PATH=/home/xuankai/anaconda3/envs/4k4d/lib/libpython3.9.so -DPython_EXECUTABLE:PATH=/home/xuankai/anaconda3/envs/4k4d/bin/python -DPython_ROOT_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d/include/python3.9 -DPython3_EXECUTABLE:PATH=/home/xuankai/anaconda3/envs/4k4d/bin/python -DPython3_ROOT_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/home/xuankai/anaconda3/envs/4k4d/include/python3.9 -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DCMAKE_BUILD_TYPE:STRING=Release
    Source directory:
      /tmp/pip-install-10dc4lcj/imgui-bundle_c98c9f82c8004e88b0d20ceb0adac3a6
    Working directory:
      /tmp/pip-install-10dc4lcj/imgui-bundle_c98c9f82c8004e88b0d20ceb0adac3a6/_skbuild/linux-x86_64-3.9/cmake-build
  Please see CMake's output for more information.

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for imgui_bundle Failed to build imgui_bundle ERROR: Could not build wheels for imgui_bundle, which is required to install pyproject.toml-based projects`

then, I tried to install imgui_bundle through https://github.com/pthom/imgui_bundle, But this asks for Python >=3.10. I wonder if you have any advice on the problems I am facing? Thank you very much!

dendenxu commented 8 months ago

Hi, thanks for your interest in our code. From the error above, it seems that you're missing a proper X11 installation:

CMake Error at /tmp/pip-build-env-wz5irisp/overlay/lib/python3.9/site-packages/cmake/data/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:230 (message): Could NOT find X11 (missing: X11_X11_INCLUDE_PATH X11_X11_LIB)

Does this machine have a GUI? Real-time visualization requires a graphical interface to function correctly. So if this a display-less server, you could try running the offline rendering commands beginning with evc -t test as listed here: https://github.com/zju3dv/4K4D?tab=readme-ov-file#rendering-of-trained-model

# Testing with input views
evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/vf0.yaml # Only render some of the view of the first frame
evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml # Only rendering some selected testing views and frames

# Rendering rotating novel views
evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/spiral.yaml,configs/specs/ibr.yaml,configs/specs/vf0.yaml # Render a static rotating novel view
evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/spiral.yaml,configs/specs/ibr.yaml # Render a dynamic rotating novel view
hhhddddddd commented 8 months ago

Thank you very much for your advice! My machine haven't a GUI, so I try executing the follow command:evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml However, I have a new error:

(4k4d) xuankai@Baicheng:~/code/4K4D$ evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml
2024-03-29 11:39:50.187596 easyvolcap.runners -> <module>: Failed to import volumetric_video_viewer.py, missing    __init__.py:10
                           package: imgui_bundle                                                                                 
2024-03-29 11:39:50.194870 __main__ -> preflight: Starting experiment: 4k4d_0013_01, command: test                     main.py:80
2024-03-29 11:39:54.231495 easyvolcap.utils.console_utils -> inner: Runtime exception: index 0 is out of     console_utils.py:391
                           bounds for axis 1 with size 0                                                                         
╭────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────╮
│ /home/xuankai/code/4K4D/easyvolcap/utils/console_utils.py:388 in inner                                                        │
│                                                                                                                               │
│ ❱ 388 │   │   │   │   return func(*args, **kwargs)                                                                            │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:272 in main                                                                │
│                                                                                                                               │
│ ❱ 272 │   else: globals()[args.type](cfg)  # invoke this (call callable_from_cfg -> call_from_cfg) I: utilize @callable_from_ │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:61 in inner                                                             │
│                                                                                                                               │
│ ❱  61 │   │   return call_from_cfg(func, cfg)                                                                                 │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg                                                     │
│                                                                                                                               │
│ ❱  52 │   return func(**call_args) # the args of function is prepared.                                                        │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:165 in test                                                                │
│                                                                                                                               │
│ ❱ 165 │   val_dataloader: "VolumetricVideoDataloader" = DATALOADERS.build(val_dataloader_cfg)  # reuse the validataion I: val │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:305 in build                                                            │
│                                                                                                                               │
│ ❱ 305 │   │   return self.build_func(*args, **kwargs, registry=self) # runner & viewer: self.build_func is build_from_cfg     │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:136 in build_from_cfg                                                   │
│                                                                                                                               │
│ ❱ 136 │   return call_from_cfg(obj_cls, args)                                                                                 │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg                                                     │
│                                                                                                                               │
│ ❱  52 │   return func(**call_args) # the args of function is prepared.                                                        │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/dataloaders/volumetric_video_dataloader.py:94 in __init__                                  │
│                                                                                                                               │
│ ❱  94 │   │   dataset: VolumetricVideoDataset = DATASETS.build(dataset_cfg)                                                   │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:305 in build                                                            │
│                                                                                                                               │
│ ❱ 305 │   │   return self.build_func(*args, **kwargs, registry=self) # runner & viewer: self.build_func is build_from_cfg     │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:136 in build_from_cfg                                                   │
│                                                                                                                               │
│ ❱ 136 │   return call_from_cfg(obj_cls, args)                                                                                 │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg                                                     │
│                                                                                                                               │
│ ❱  52 │   return func(**call_args) # the args of function is prepared.                                                        │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/dataloaders/datasets/image_based_dataset.py:56 in __init__                                 │
│                                                                                                                               │
│ ❱  56 │   │   call_from_cfg(super().__init__, kwargs, skip_loading_images=skip_loading_images)                                │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg                                                     │
│                                                                                                                               │
│ ❱  52 │   return func(**call_args) # the args of function is prepared.                                                        │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py:276 in __init__                           │
│                                                                                                                               │
│ ❱  276 │   │   self.load_paths()  # load image files into self.ims                                                            │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py:382 in load_paths                         │
│                                                                                                                               │
│ ❱  382 │   │   self.ims_dir = join(*split(dirname(self.ims[0, 0]))[:-1])  # logging only                                      │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: index 0 is out of bounds for axis 1 with size 0
*** index 0 is out of bounds for axis 1 with size 0
> /home/xuankai/code/4K4D/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py(382)load_paths()
    380         ims = [np.asarray(ims[i])[:min([len(i) for i in ims])] for i in range(len(ims))]  # deal with the fact that some 
    381         self.ims = np.asarray(ims)  # V, N
--> 382         self.ims_dir = join(*split(dirname(self.ims[0, 0]))[:-1])  # logging only
    383 
    384         # TypeError: can't convert np.ndarray of type numpy.str_. The only supported types are: float64, float32, float16

(Pdbr) exit

I wonder if you have any advice on the problems I am facing? Thank you very much!

dendenxu commented 8 months ago

Looks like the code could not find your dataset, please check whether join(self.data_root, self.images_dir) exists. The dataset structure for EasyVolcap is documented here: https://github.com/zju3dv/4K4D?tab=readme-ov-file#datasets

hhhddddddd commented 8 months ago

Thank you very much for helping me find the cause of the above problem！ I encountered a new problem when executing evc -t test -c configs/projects/realtime4dv/rendering/4k4d_0013_01.yaml,configs/specs/eval.yaml,configs/specs/vf0.yaml

2024-03-29 15:21:35.520479 easyvolcap.utils.console_utils -> inner: Runtime exception: The size of tensor a  console_utils.py:391
                           (203840) must match the size of tensor b (440608) at non-singleton dimension 1                        
╭────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────╮
│ /home/xuankai/code/4K4D/easyvolcap/utils/console_utils.py:388 in inner                                                        │
│                                                                                                                               │
│ ❱ 388 │   │   │   │   return func(*args, **kwargs)                                                                            │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:272 in main                                                                │
│                                                                                                                               │
│ ❱ 272 │   else: globals()[args.type](cfg)  # invoke this (call callable_from_cfg -> call_from_cfg) I: utilize @callable_from_ │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:61 in inner                                                             │
│                                                                                                                               │
│ ❱  61 │   │   return call_from_cfg(func, cfg)                                                                                 │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/engine/registry.py:52 in call_from_cfg                                                     │
│                                                                                                                               │
│ ❱  52 │   return func(**call_args) # the args of function is prepared.                                                        │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:181 in test                                                                │
│                                                                                                                               │
│ ❱ 181 │   launcher(**kwargs, runner_function=runner.test, runner_object=runner)                                               │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/scripts/main.py:52 in launcher                                                             │
│                                                                                                                               │
│ ❱  52 │   runner_function()                                                                                                   │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/runners/volumetric_video_runner.py:275 in test                                             │
│                                                                                                                               │
│ ❱ 275 │   │   self.test_epoch(epoch)                                                                                          │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/runners/volumetric_video_runner.py:423 in test_epoch                                       │
│                                                                                                                               │
│ ❱ 423 │   │   for _ in test_generator: pass  # the actual calling                                                             │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/runners/volumetric_video_runner.py:435 in test_generator                                   │
│                                                                                                                               │
│ ❱ 435 │   │   │   │   scalar_stats = self.evaluator.evaluate(output, batch)                                                   │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/runners/evaluators/volumetric_video_evaluator.py:31 in evaluate                            │
│                                                                                                                               │
│ ❱ 31 │   │   img, img_gt, _ = super().generate_type(output, batch, Visualization.RENDER)                                      │
│                                                                                                                               │
│ /home/xuankai/code/4K4D/easyvolcap/runners/visualizers/volumetric_video_visualizer.py:176 in generate_type                    │
│                                                                                                                               │
│ ❱ 176 │   │   │   img_gt = img_gt + output.bg_color * (1 - batch.msk)                                                         │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: The size of tensor a (203840) must match the size of tensor b (440608) at non-singleton dimension 1
2024-03-29 15:21:35.806174 easyvolcap.runners.volumetric…    0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/3  0:00:18 < -:--:--   ?  it/s v…
                           -> test_generator:                                                                                    
*** The size of tensor a (203840) must match the size of tensor b (440608) at non-singleton dimension 1
> /home/xuankai/code/4K4D/easyvolcap/runners/visualizers/volumetric_video_visualizer.py(176)generate_type()
    174         if img_gt is not None and 'bg_color' in output and 'msk' in batch:
    175             # Fill gt with input BG colors
--> 176             img_gt = img_gt + output.bg_color * (1 - batch.msk)
    177 
    178         if self.store_image_error and img_gt is not None:

(Pdbr)

I wonder if you have any advice on the problems I am facing? In addition, I would also like to ask about the video memory requirements of the NVIDIA 3090 for the experiment in 4k4d. Thank you very much!

dendenxu commented 8 months ago

Strangely, I'm unable to reproduce the errors on my end. Can you help me make sure the pre-trained model is loaded? There should be a line like: easyvolcap.utils.net_utils -> load_network: Loaded network data/trained_model/4k4d_0013_01/1599.npz at epoch -1 in your log.

For training, 4K4D shouldn't require more than 5GB of VRAM for the DNA-Rendering dataset.