search

skhu101 / GauHuman

Code for our CVPR'2024 paper "GauHuman: Articulated Gaussian Splatting from Monocular Human Videos"
Other
335 stars 28 forks source link

Questions about annots.npy and the Training process #39

Closed YujiaLin-523 closed 3 months ago

YujiaLin-523 commented 3 months ago

Firstly, I want to thank you for your nice work. However, I am puzzled by three questions when I try to run your code.

1. No such file in ... annots.npy That means the ZJU-MoCap Dataset I downloaded here following the guide of instant-nvr does not contain annnots.npy in some folders. Then I just rename annots_new.npy into annots.npy to solve the error. I am not sure if this solution is right because I met other two problems after that. This is similar to #3. @skhu101 Can you tell me if it is OK?

2. ValueError: not enough values to unpack (expected 4, got 2) This happened to my_377 after I renamed annots_new.npy. The complete output of the terminal is as follows. How can I solve this?

/home/ubuntu/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:1967: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
  warnings.warn(
Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
/home/ubuntu/.local/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/home/ubuntu/.local/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Loading model from: /home/ubuntu/.local/lib/python3.8/site-packages/lpips/weights/v0.1/vgg.pth
Optimizing 
Output folder: ./output/zju_mocap_refine/my_377_100_pose_correction_lbs_offset_split_clone_merge_prune [12/07 12:47:50]
Found annots.json file, assuming ZJU_MoCap_refine data set! [12/07 12:47:50]
Reading Training Transforms [12/07 12:47:50]
Reading Test Transforms [12/07 12:47:58]
Loading Training Cameras [12/07 12:48:28]
Loading Test Cameras [12/07 12:48:31]
Number of points at initialisation :  6890 [12/07 12:48:33]
Training progress:   0%|                                                                                                                                                                                                                                                             | 0/1200 [00:00<?, ?it/s]Traceback (most recent call last):
  File "train.py", line 301, in <module>
    training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
  File "train.py", line 104, in training
    render_pkg = render(viewpoint_cam, gaussians, pipe, background)
  File "/home/ubuntu/lyj/Project/GauHuman/gaussian_renderer/__init__.py", line 111, in render
    rendered_image, radii, depth, alpha = rasterizer(
ValueError: not enough values to unpack (expected 4, got 2)
Training progress:   0%|

3. IndexError: index 0 is out of bounds for axis 0 with size 0 It is very interesting that some folders in ZJU_MoCap have this problem and others have problem 2. For example, my_386 has this problem and the complete information is as follows:

/home/ubuntu/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:1967: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
  warnings.warn(
Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
/home/ubuntu/.local/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/home/ubuntu/.local/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Loading model from: /home/ubuntu/.local/lib/python3.8/site-packages/lpips/weights/v0.1/vgg.pth
Optimizing 
Output folder: ./output/zju_mocap_refine/my_386_100_pose_correction_lbs_offset_split_clone_merge_prune [12/07 12:48:43]
Found annots.json file, assuming ZJU_MoCap_refine data set! [12/07 12:48:43]
Reading Training Transforms [12/07 12:48:43]
Traceback (most recent call last):
  File "train.py", line 301, in <module>
    training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
  File "train.py", line 45, in training
    scene = Scene(dataset, gaussians)
  File "/home/ubuntu/lyj/Project/GauHuman/scene/__init__.py", line 52, in __init__
    scene_info = sceneLoadTypeCallbacks["ZJU_MoCap_refine"](args.source_path, args.white_background, args.exp_name, args.eval)
  File "/home/ubuntu/lyj/Project/GauHuman/scene/dataset_readers.py", line 704, in readZJUMoCapRefineInfo
    train_cam_infos = readCamerasZJUMoCapRefine(path, train_view, white_background, split='train')
  File "/home/ubuntu/lyj/Project/GauHuman/scene/dataset_readers.py", line 611, in readCamerasZJUMoCapRefine
    image_path = os.path.join(path, ims[pose_index][view_index].replace('\\', '/'))
IndexError: index 0 is out of bounds for axis 0 with size 0

These problems may be common for your paper readers, so I am looking forward to your kind response!

skhu101 commented 3 months ago

Hi, thanks for your interest in our work. 1. If you get the link from the authors of InstantAvatar, the refined ZJU_MoCap should contain both annot.npy and annot_new.npy. Please ensure you are using the refined ZJU_MoCap dataset, a refined version of the ZJU_MoCap dataset with refined camera and SMPL parameters. 2. Please check whether you are using diff-gaussian-rasterization from https://github.com/ashawkey/diff-gaussian-rasterization. Based on the official version, we implement the alpha mask loss version. 3. For refined ZJU_MoCap, we select 6 human subjects (377, 386, 387, 392, 393, 394) to conduct experiments. If you use the refined ZJU_MoCap dataset, this issue could be solved. Could you check about it? If you have further problems, feel free to let me know.

YujiaLin-523 commented 3 months ago

@skhu101 Thanks for your guidance. I downloaded the ZJU_MoCap Refine Dataset again and launched the training process. But it still shows that there is problem associated with annots.npy. Here is an example:

/home/ubuntu/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py:1967: UserWarning: TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation. 
If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'].
  warnings.warn(
Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off]
/home/ubuntu/.local/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/home/ubuntu/.local/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=VGG16_Weights.IMAGENET1K_V1`. You can also use `weights=VGG16_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Loading model from: /home/ubuntu/.local/lib/python3.8/site-packages/lpips/weights/v0.1/vgg.pth
Optimizing 
Output folder: ./output/zju_mocap_refine/my_394_100_pose_correction_lbs_offset_split_clone_merge_prune [15/07 19:27:08]
Found annots.json file, assuming ZJU_MoCap_refine data set! [15/07 19:27:08]
Reading Training Transforms [15/07 19:27:08]
Traceback (most recent call last):
  File "train.py", line 301, in <module>
    training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from)
  File "train.py", line 45, in training
    scene = Scene(dataset, gaussians)
  File "/home/ubuntu/lyj/Project/GauHuman/scene/__init__.py", line 52, in __init__
    scene_info = sceneLoadTypeCallbacks["ZJU_MoCap_refine"](args.source_path, args.white_background, args.exp_name, args.eval)
  File "/home/ubuntu/lyj/Project/GauHuman/scene/dataset_readers.py", line 704, in readZJUMoCapRefineInfo
    train_cam_infos = readCamerasZJUMoCapRefine(path, train_view, white_background, split='train')
  File "/home/ubuntu/lyj/Project/GauHuman/scene/dataset_readers.py", line 562, in readCamerasZJUMoCapRefine
    annots = np.load(ann_file, allow_pickle=True).item()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/numpy/lib/npyio.py", line 405, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/lyj/Project/GauHuman/data/zju_mocap_refine/my_394/annots.npy'

What should I do next to solve it?

skhu101 commented 3 months ago

Hi, this error shows no annots.npy in your downloaded refined ZJU_MoCap folder. Attached is the screen shoot of my downloaded folder. WechatIMG932

YujiaLin-523 commented 3 months ago

I think I know why this problem happened. I use Ubuntu 20.04.6 and run your code in a Docker container. So there may be some differences when unzipping this dataset in my special environment, leading to the error. I'll try again on other OS then and here is my folder's screenshot, which can be a reference for other readers. image