search

zju3dv / 4K4D

[CVPR 2024] 4K4D: Real-Time 4D View Synthesis at 4K Resolution
https://zju3dv.github.io/4k4d/
Other
1.55k stars 66 forks source link

Error during rendering #21

Open rexainn opened 7 months ago

rexainn commented 7 months ago

Command: evc -t test -c configs/projects/realtime4dv/rendering/4k4d_rxy_demo.yaml,configs/specs/eval.yaml,configs/specs/spiral.yaml,configs/specs/ibr.yaml,configs/specs/vf0.yaml

Error如下：

Error log ```shell 2024-02-01 22:51:17.099045 easyvolcap.runners -> : Failed to import submodule volumetric_video_viewer of __init__.py:9 /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/runners/__init__.py ╭──────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────╮ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/runners/__init__.py:6 in │ │ │ │ ❱ 6 │ │ exec(f'from . import {module}') │ │ in :1 │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/runners/volumetric_video_viewer.py:21 in │ │ │ │ ❱ 21 from imgui_bundle import imgui_color_text_edit as ed │ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ModuleNotFoundError: No module named 'imgui_bundle' 2024-02-01 22:51:17.143225 easyvolcap.scripts.main -> preflight: Starting experiment: 4k4d_rxy_demo, command: test main.py:80 2024-02-01 easyvolca… Loading mask bytes for data/rxy/rxy_demo/masks VAL 100% ━━━━━━━━━━━ 4/4 0:00:00 < 0:00:00 258.9 it/s p… 22:51:17.… -> load_resi… 2024-02-01 easyvolca… Loading imgs bytes for data/rxy/rxy_demo/images VAL 100% ━━━━━━━━━━ 4/4 0:00:00 < 0:00:00 403.3 it/s p… 22:51:18.… -> load_resi… 2024-02-01 22:51:18.886432 easyvolcap.utils.console_utils -> inner: Runtime exception: OpenCV(4.9.0) console_utils.py:341 /io/opencv/modules/imgcodecs/src/loadsave.cpp:1121: error: (-215:Assertion failed) !image.empty() in function 'imencode' ╭──────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────╮ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/console_utils.py:338 in inner │ │ │ │ ❱ 338 │ │ │ return func(*args, **kwargs) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:56 in inner │ │ │ │ ❱ 56 │ │ return call_from_cfg(func, cfg) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg │ │ │ │ ❱ 47 │ return func(**call_args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/scripts/main.py:158 in test │ │ │ │ ❱ 158 │ val_dataloader: "VolumetricVideoDataloader" = DATALOADERS.build(val_dataloader_cfg) # reuse the validataion │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:300 in build │ │ │ │ ❱ 300 │ │ return self.build_func(*args, **kwargs, registry=self) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:131 in build_from_cfg │ │ │ │ ❱ 131 │ return call_from_cfg(obj_cls, args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg │ │ │ │ ❱ 47 │ return func(**call_args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/volumetric_video_dataloader.py:94 in __init__ │ │ │ │ ❱ 94 │ │ dataset: VolumetricVideoDataset = DATASETS.build(dataset_cfg) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:300 in build │ │ │ │ ❱ 300 │ │ return self.build_func(*args, **kwargs, registry=self) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:131 in build_from_cfg │ │ │ │ ❱ 131 │ return call_from_cfg(obj_cls, args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg │ │ │ │ ❱ 47 │ return func(**call_args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/image_based_inference_dataset.py:33 i │ │ │ │ ❱ 33 │ │ call_from_cfg(super().__init__, kwargs) # will have prepared other parts of the dataset (interpolation or o │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg │ │ │ │ ❱ 47 │ return func(**call_args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/volumetric_video_inference_dataset.py │ │ │ │ ❱ 56 │ │ call_from_cfg(super().__init__, kwargs) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg │ │ │ │ ❱ 47 │ return func(**call_args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py:272 in __ │ │ │ │ ❱ 272 │ │ │ self.load_bytes() # load image bytes (also load vhulls) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/image_based_inference_dataset.py:70 i │ │ │ │ ❱ 70 │ │ return VolumetricVideoDataset.load_bytes(self) # store images │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py:472 in lo │ │ │ │ ❱ 472 │ │ │ │ decode_crop_fill_ims_bytes(self.ims_bytes, self.mks_bytes, self.Ks.numpy(), self.Rs.numpy(), self.T │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/data_utils.py:1679 in decode_crop_fill_ims_bytes │ │ │ │ ❱ 1679 │ out = parallel_execution(list(ims_bytes), list(mks_bytes), list(Ks), list(Rs), list(Ts), list(bounds), │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/parallel_utils.py:69 in parallel_execution │ │ │ │ ❱ 69 │ │ │ │ results.append(async_result.get()) # will sync the corresponding thread │ │ │ │ /home/jj/.conda/envs/sadtalker/lib/python3.8/multiprocessing/pool.py:771 in get │ │ │ │ ❱ 771 │ │ │ raise self._value │ │ │ │ /home/jj/.conda/envs/sadtalker/lib/python3.8/multiprocessing/pool.py:125 in worker │ │ │ │ ❱ 125 │ │ │ result = (True, func(*args, **kwds)) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/data_utils.py:1663 in decode_crop_fill_im_bytes │ │ │ │ ❱ 1663 │ im_bytes = cv2.imencode(encode_ext, img, [cv2.IMWRITE_JPEG_QUALITY, jpeg_quality, cv2.IMWRITE_PNG_COMPRESSION, │ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ error: OpenCV(4.9.0) /io/opencv/modules/imgcodecs/src/loadsave.cpp:1121: error: (-215:Assertion failed) !image.empty() in function 'imencode' 2024-02-01 easyvolcap.… Cropping msks imgs for data/rxy/rxy_demo VAL 25% ━━━╺━━━━━━━━━ 1/4 0:00:01 < -:--:-- ? it/s p… 22:51:19.806… -> decode_crop… *** OpenCV(4.9.0) /io/opencv/modules/imgcodecs/src/loadsave.cpp:1121: error: (-215:Assertion failed) !image.empty() in function 'imencode' > /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/data_utils.py(1663)decode_crop_fill_im_bytes() 1661 K[1, 2] -= y 1662 -> 1663 im_bytes = cv2.imencode(encode_ext, img, [cv2.IMWRITE_JPEG_QUALITY, jpeg_quality, cv2.IMWRITE_PNG_COMPRESSION, png 1664 mk_bytes = cv2.imencode(encode_ext, msk, [cv2.IMWRITE_JPEG_QUALITY, jpeg_quality, cv2.IMWRITE_PNG_COMPRESSION, png 1665 return im_bytes, mk_bytes, K, h, w, x, y ```
rexainn commented 7 months ago

如果把mask.yaml注释掉，又会有新的error：

Error log: ```shell 2024-02-01 22:56:16.214839 easyvolcap.runners -> : Failed to import submodule volumetric_video_viewer of __init__.py:9 /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/runners/__init__.py ╭──────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────╮ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/runners/__init__.py:6 in │ │ │ │ ❱ 6 │ │ exec(f'from . import {module}') │ │ in :1 │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/runners/volumetric_video_viewer.py:21 in │ │ │ │ ❱ 21 from imgui_bundle import imgui_color_text_edit as ed │ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ModuleNotFoundError: No module named 'imgui_bundle' 2024-02-01 22:56:16.261401 easyvolcap.scripts.main -> preflight: Starting experiment: 4k4d_rxy_demo, command: test main.py:80 2024-02-01 easyvolca… Loading imgs bytes for data/rxy/rxy_demo/images VAL 100% ━━━━━━━━━━ 4/4 0:00:00 < 0:00:00 126.4 it/s p… 22:56:17.… -> load_resi… 2024-02-01 22:56:19.341804 easyvolcap.utils.console_utils -> inner: Runtime exception: 'SuperChargedR4DV console_utils.py:341 is not in the samplers registry' ╭──────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────╮ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/console_utils.py:338 in inner │ │ │ │ ❱ 338 │ │ │ return func(*args, **kwargs) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:56 in inner │ │ │ │ ❱ 56 │ │ return call_from_cfg(func, cfg) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg │ │ │ │ ❱ 47 │ return func(**call_args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/scripts/main.py:160 in test │ │ │ │ ❱ 160 │ model: "VolumetricVideoModel" = MODELS.build(model_cfg) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:300 in build │ │ │ │ ❱ 300 │ │ return self.build_func(*args, **kwargs, registry=self) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:131 in build_from_cfg │ │ │ │ ❱ 131 │ return call_from_cfg(obj_cls, args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg │ │ │ │ ❱ 47 │ return func(**call_args) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/models/volumetric_video_model.py:58 in __init__ │ │ │ │ ❱ 58 │ │ self.sampler: ImportanceSampler = SAMPLERS.build(sampler_cfg, network=self.network) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:300 in build │ │ │ │ ❱ 300 │ │ return self.build_func(*args, **kwargs, registry=self) │ │ │ │ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:121 in build_from_cfg │ │ │ │ ❱ 121 │ │ │ raise KeyError( │ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ KeyError: 'SuperChargedR4DV is not in the samplers registry' *** 'SuperChargedR4DV is not in the samplers registry' > /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py(121)build_from_cfg() 119 obj_cls = registry.get(obj_type) 120 if obj_cls is None: --> 121 raise KeyError( 122 f'{obj_type} is not in the {registry.name} registry') 123 elif inspect.isclass(obj_type) or inspect.isfunction(obj_type): ```
dendenxu commented 7 months ago

您好，从error log来看，是没装imgui_bundle这个包

rexainn commented 7 months ago

但是我使用的是evc -t test呀，ingui是必装的吗

dendenxu commented 7 months ago

因吹斯汀，我尝试复现一下

dendenxu commented 7 months ago

Ah，和imgui没关系，原因是你之前装过easyvolcap，需要在4k4d仓库重新跑一下pip install -e .指令（readme里的installation section有提及），否则会引用easyvolcap里的代码，自然找不到SuperChargedR4DV

rexainn commented 7 months ago

Command: evc -t test -c configs/projects/realtime4dv/rendering/4k4d_rxy_demo.yaml,configs/specs/eval.yaml,configs/specs/spiral.yaml,configs/specs/ibr.yaml,configs/specs/vf0.yaml

Error如下：

Error log

2024-02-01 22:51:17.099045 easyvolcap.runners -> <module>: Failed to import submodule volumetric_video_viewer of __init__.py:9
                           /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/runners/__init__.py
╭──────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────╮
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/runners/__init__.py:6 in <module>                          │
│                                                                                                                            │
│ ❱  6 │   │   exec(f'from . import {module}')                                                                               │
│ in <module>:1                                                                                                              │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/runners/volumetric_video_viewer.py:21 in <module>          │
│                                                                                                                            │
│ ❱   21 from imgui_bundle import imgui_color_text_edit as ed                                                                │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'imgui_bundle'
2024-02-01 22:51:17.143225 easyvolcap.scripts.main -> preflight: Starting experiment: 4k4d_rxy_demo, command: test  main.py:80
2024-02-01 easyvolca… Loading mask bytes for data/rxy/rxy_demo/masks VAL 100% ━━━━━━━━━━━ 4/4  0:00:00 < 0:00:00 258.9 it/s p…
22:51:17.… ->
           load_resi…
2024-02-01 easyvolca… Loading imgs bytes for data/rxy/rxy_demo/images VAL 100% ━━━━━━━━━━ 4/4  0:00:00 < 0:00:00 403.3 it/s p…
22:51:18.… ->
           load_resi…
2024-02-01 22:51:18.886432 easyvolcap.utils.console_utils -> inner: Runtime exception: OpenCV(4.9.0)      console_utils.py:341
                           /io/opencv/modules/imgcodecs/src/loadsave.cpp:1121: error: (-215:Assertion
                           failed) !image.empty() in function 'imencode'

╭──────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────╮
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/console_utils.py:338 in inner                        │
│                                                                                                                            │
│ ❱ 338 │   │   │   return func(*args, **kwargs)                                                                             │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:56 in inner                             │
│                                                                                                                            │
│ ❱  56 │   │   return call_from_cfg(func, cfg)                                                                              │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg                     │
│                                                                                                                            │
│ ❱  47 │   return func(**call_args)                                                                                         │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/scripts/main.py:158 in test                                │
│                                                                                                                            │
│ ❱ 158 │   val_dataloader: "VolumetricVideoDataloader" = DATALOADERS.build(val_dataloader_cfg)  # reuse the validataion     │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:300 in build                            │
│                                                                                                                            │
│ ❱ 300 │   │   return self.build_func(*args, **kwargs, registry=self)                                                       │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:131 in build_from_cfg                   │
│                                                                                                                            │
│ ❱ 131 │   return call_from_cfg(obj_cls, args)                                                                              │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg                     │
│                                                                                                                            │
│ ❱  47 │   return func(**call_args)                                                                                         │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/volumetric_video_dataloader.py:94 in __init__  │
│                                                                                                                            │
│ ❱  94 │   │   dataset: VolumetricVideoDataset = DATASETS.build(dataset_cfg)                                                │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:300 in build                            │
│                                                                                                                            │
│ ❱ 300 │   │   return self.build_func(*args, **kwargs, registry=self)                                                       │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:131 in build_from_cfg                   │
│                                                                                                                            │
│ ❱ 131 │   return call_from_cfg(obj_cls, args)                                                                              │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg                     │
│                                                                                                                            │
│ ❱  47 │   return func(**call_args)                                                                                         │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/image_based_inference_dataset.py:33 i │
│                                                                                                                            │
│ ❱  33 │   │   call_from_cfg(super().__init__, kwargs)  # will have prepared other parts of the dataset (interpolation or o │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg                     │
│                                                                                                                            │
│ ❱  47 │   return func(**call_args)                                                                                         │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/volumetric_video_inference_dataset.py │
│                                                                                                                            │
│ ❱  56 │   │   call_from_cfg(super().__init__, kwargs)                                                                      │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/engine/registry.py:47 in call_from_cfg                     │
│                                                                                                                            │
│ ❱  47 │   return func(**call_args)                                                                                         │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py:272 in __ │
│                                                                                                                            │
│ ❱  272 │   │   │   self.load_bytes()  # load image bytes (also load vhulls)                                                │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/image_based_inference_dataset.py:70 i │
│                                                                                                                            │
│ ❱  70 │   │   return VolumetricVideoDataset.load_bytes(self)  # store images                                               │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/dataloaders/datasets/volumetric_video_dataset.py:472 in lo │
│                                                                                                                            │
│ ❱  472 │   │   │   │   decode_crop_fill_ims_bytes(self.ims_bytes, self.mks_bytes, self.Ks.numpy(), self.Rs.numpy(), self.T │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/data_utils.py:1679 in decode_crop_fill_ims_bytes     │
│                                                                                                                            │
│ ❱ 1679 │   out = parallel_execution(list(ims_bytes), list(mks_bytes), list(Ks), list(Rs), list(Ts), list(bounds),          │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/parallel_utils.py:69 in parallel_execution           │
│                                                                                                                            │
│ ❱ 69 │   │   │   │   results.append(async_result.get())  # will sync the corresponding thread                              │
│                                                                                                                            │
│ /home/jj/.conda/envs/sadtalker/lib/python3.8/multiprocessing/pool.py:771 in get                                            │
│                                                                                                                            │
│ ❱ 771 │   │   │   raise self._value                                                                                        │
│                                                                                                                            │
│ /home/jj/.conda/envs/sadtalker/lib/python3.8/multiprocessing/pool.py:125 in worker                                         │
│                                                                                                                            │
│ ❱ 125 │   │   │   result = (True, func(*args, **kwds))                                                                     │
│                                                                                                                            │
│ /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/data_utils.py:1663 in decode_crop_fill_im_bytes      │
│                                                                                                                            │
│ ❱ 1663 │   im_bytes = cv2.imencode(encode_ext, img, [cv2.IMWRITE_JPEG_QUALITY, jpeg_quality, cv2.IMWRITE_PNG_COMPRESSION,  │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
error: OpenCV(4.9.0) /io/opencv/modules/imgcodecs/src/loadsave.cpp:1121: error: (-215:Assertion failed) !image.empty() in function 'imencode'

2024-02-01    easyvolcap.… Cropping msks imgs for data/rxy/rxy_demo VAL  25% ━━━╺━━━━━━━━━ 1/4  0:00:01 < -:--:--   ?  it/s p…
22:51:19.806… ->
              decode_crop…
*** OpenCV(4.9.0) /io/opencv/modules/imgcodecs/src/loadsave.cpp:1121: error: (-215:Assertion failed) !image.empty() in
function 'imencode'

> /DATA/DATA1/rxy_code/projects/202312_demo/EasyVolcap/easyvolcap/utils/data_utils.py(1663)decode_crop_fill_im_bytes()
   1661     K[1, 2] -= y
   1662
-> 1663     im_bytes = cv2.imencode(encode_ext, img, [cv2.IMWRITE_JPEG_QUALITY, jpeg_quality, cv2.IMWRITE_PNG_COMPRESSION, png
   1664     mk_bytes = cv2.imencode(encode_ext, msk, [cv2.IMWRITE_JPEG_QUALITY, jpeg_quality, cv2.IMWRITE_PNG_COMPRESSION, png
   1665     return im_bytes, mk_bytes, K, h, w, x, y

您好，但是这个error还是没有解决。

rexainn commented 7 months ago

不使用mask会报：

IndexError: list index out of range
2024-02-04 18:11:23.256265 easyvolcap.runners.volumet…    0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/300  0:00:02 < -:--:--   ?  it/s v…
                           -> test_generator:
*** list index out of range
> /DATA/DATA1/rxy_code/projects/202312_demo/4K4D/easyvolcap/models/samplers/super_charged_r4dv.py(492)<listcomp>()
    490         # Extract input
    491         xyz = torch.stack([self.pcds[l] for l in index])  # B, N, 3
--> 492         rad = torch.stack([self.rads[l] for l in index])  # B, N, 3
    493         occ = torch.stack([self.occs[l] for l in index])  # B, N, 3
    494         cent = torch.stack([self.cents[l] for l in index])  # B, S, 3
dendenxu commented 7 months ago

你好，4k4d现在的代码不支持training，所以会默认load trained models，load不到就会报你注释掉mask之后的错误。

rexainn commented 7 months ago

evc -t test -c configs/projects/realtime4dv/rendering/4k4d_rxy_demo.yaml,configs/specs/eval.yaml,configs/specs/spiral.yaml,configs/specs/ibr.yaml,configs/specs/vf0.yaml 这行指令的意思是使用预训练模型test我的数据集嘛。那我是要怎么指定预训练模型？

dendenxu commented 7 months ago

你好，我们的readme里有介绍详细的渲染流程。

rexainn commented 7 months ago

我是按照readme做的，trained model都下载在指定路径里了TT我和readme唯一的区别就是使用了自己的数据集，所以重新写了config。

问题是在custom dataset上test pretrained model，他应该从data/trained_model/4k4d_0013_01加载模型，而不是从data/trained_model/4k4d_rxy_demo加载模型。

看起来是没有像EasyVolcap一样有inference的代码。

dendenxu commented 7 months ago

是的，4k4d是per-scene optimization方法