search

hustvl / 4DGaussians

[CVPR 2024] 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
https://guanjunwu.github.io/4dgs/
Apache License 2.0
2.26k stars 189 forks source link

Error when Training Hypernerf Dataset #58

Closed KevinYu0910 closed 10 months ago

KevinYu0910 commented 11 months ago

I have encountered an issue while attempting to train the hypernerf dataset. Here are the details:

I have successfully executed the following command:

python train.py -s data/dnerf/bouncingballs --port 6017 --expname "dnerf/bouncingballs" --configs arguments/dnerf/bouncingballs.py

However, when I tried to train the hypernerf dataset with the following command:

python train.py -s data/hypernerf/vrig/broom2 --port 6017 --expname "hypernerf/broom2" --config arguments/hypernerf/broom2.py

I encountered the following error:

Traceback (most recent call last):
  File "train.py", line 410, in <module>
    training(lp.extract(args), hp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from, args.expname)
  File "train.py", line 284, in training
    scene = Scene(dataset, gaussians, load_coarse=None)
  File "/data/data_2t/4DGaussians/scene/__init__.py", line 56, in __init__
    scene_info = sceneLoadTypeCallbacks["nerfies"](args.source_path, False, args.eval)
  File "/data/data_2t/4DGaussians/scene/dataset_readers.py", line 386, in readHyperDataInfos
    pcd = fetchPly(ply_path)
  File "/data/data_2t/4DGaussians/scene/dataset_readers.py", line 125, in fetchPly
    plydata = PlyData.read(path)
  File "/home/kevin/.local/lib/python3.8/site-packages/plyfile.py", line 158, in read
    (must_close, stream) = _open_stream(stream, 'read')
  File "/home/kevin/.local/lib/python3.8/site-packages/plyfile.py", line 1345, in _open_stream
    return (True, open(stream, read_or_write[0] + 'b'))
FileNotFoundError: [Errno 2] No such file or directory: '/data/data_2t/4DGaussians/data/hypernerf/vrig/broom2/points3D_downsample.ply'

I would appreciate it if you could provide guidance on any missing steps or potential solutions for this issue. Thank you.

guanjunwu commented 11 months ago

You can follow colmap.sh to generate point clouds firstly. bash colmap.sh data/hypernerf/virg/broom2 hypernerf

kevinyu-0910 commented 11 months ago

When I executed bash colmap.sh data/hypernerf/virg/broom2 hypernerf I encountered the following issue.

Writing reconstruction...
Writing configuration...
Writing scripts...
Elapsed time: 0.063 [minutes]
ERROR: Dense stereo reconstruction requires CUDA, which is not available on your system.

StereoFusion::Options
---------------------
max_image_size: -1
min_num_pixels: 5
max_num_pixels: 10000
max_traversal_depth: 100
max_reproj_error: 2
max_depth_error: 0.01
max_normal_error: 10
check_num_images: 50
cache_size: 32

Reading workspace...
Reading configuration...
WARNING: Ignoring image right1_000056.png, because input does not exist.
WARNING: Ignoring image right1_000054.png, because input does not exist.
WARNING: Ignoring image right1_000052.png, because input does not exist.
WARNING: Ignoring image right1_000050.png, because input does not exist.
WARNING: Ignoring image right1_000048.png, because input does not exist.
WARNING: Ignoring image right1_000046.png, because input does not exist.
WARNING: Ignoring image right1_000044.png, because input does not exist.
WARNING: Ignoring image right1_000042.png, because input does not exist.
WARNING: Ignoring image right1_000040.png, because input does not exist.
WARNING: Ignoring image right1_000038.png, because input does not exist.
WARNING: Ignoring image right1_000036.png, because input does not exist.
WARNING: Ignoring image right1_000034.png, because input does not exist.
WARNING: Ignoring image right1_000032.png, because input does not exist.
WARNING: Ignoring image right1_000030.png, because input does not exist.
WARNING: Ignoring image right1_000028.png, because input does not exist.
WARNING: Ignoring image right1_000026.png, because input does not exist.
WARNING: Ignoring image right1_000024.png, because input does not exist.
WARNING: Ignoring image right1_000022.png, because input does not exist.
WARNING: Ignoring image right1_000020.png, because input does not exist.
WARNING: Ignoring image right1_000018.png, because input does not exist.
WARNING: Ignoring image right1_000016.png, because input does not exist.
WARNING: Ignoring image right1_000014.png, because input does not exist.
WARNING: Ignoring image right1_000012.png, because input does not exist.
WARNING: Ignoring image right1_000010.png, because input does not exist.
WARNING: Ignoring image right1_000008.png, because input does not exist.
WARNING: Ignoring image right1_000006.png, because input does not exist.
WARNING: Ignoring image right1_000004.png, because input does not exist.
WARNING: Ignoring image right1_000002.png, because input does not exist.
WARNING: Ignoring image right1_000000.png, because input does not exist.
WARNING: Ignoring image left1_000195.png, because input does not exist.
WARNING: Ignoring image left1_000193.png, because input does not exist.
WARNING: Ignoring image left1_000191.png, because input does not exist.
WARNING: Ignoring image left1_000189.png, because input does not exist.
WARNING: Ignoring image left1_000187.png, because input does not exist.
WARNING: Ignoring image left1_000185.png, because input does not exist.
WARNING: Ignoring image left1_000183.png, because input does not exist.
WARNING: Ignoring image left1_000181.png, because input does not exist.
WARNING: Ignoring image left1_000179.png, because input does not exist.
WARNING: Ignoring image left1_000177.png, because input does not exist.
WARNING: Ignoring image left1_000175.png, because input does not exist.
WARNING: Ignoring image left1_000173.png, because input does not exist.
WARNING: Ignoring image left1_000171.png, because input does not exist.
WARNING: Ignoring image left1_000169.png, because input does not exist.
WARNING: Ignoring image left1_000167.png, because input does not exist.
WARNING: Ignoring image left1_000165.png, because input does not exist.
WARNING: Ignoring image left1_000163.png, because input does not exist.
WARNING: Ignoring image left1_000161.png, because input does not exist.
WARNING: Ignoring image left1_000159.png, because input does not exist.
WARNING: Ignoring image left1_000157.png, because input does not exist.
WARNING: Ignoring image left1_000155.png, because input does not exist.
WARNING: Ignoring image left1_000153.png, because input does not exist.
WARNING: Ignoring image left1_000151.png, because input does not exist.
WARNING: Ignoring image left1_000149.png, because input does not exist.
WARNING: Ignoring image left1_000147.png, because input does not exist.
WARNING: Ignoring image left1_000145.png, because input does not exist.
WARNING: Ignoring image left1_000143.png, because input does not exist.
WARNING: Ignoring image left1_000141.png, because input does not exist.
WARNING: Ignoring image left1_000139.png, because input does not exist.
WARNING: Ignoring image left1_000137.png, because input does not exist.
WARNING: Ignoring image left1_000135.png, because input does not exist.
WARNING: Ignoring image left1_000133.png, because input does not exist.
WARNING: Ignoring image left1_000131.png, because input does not exist.
WARNING: Ignoring image left1_000129.png, because input does not exist.
WARNING: Ignoring image left1_000127.png, because input does not exist.
WARNING: Ignoring image left1_000125.png, because input does not exist.
WARNING: Ignoring image left1_000123.png, because input does not exist.
WARNING: Ignoring image left1_000121.png, because input does not exist.
WARNING: Ignoring image left1_000119.png, because input does not exist.
WARNING: Ignoring image left1_000057.png, because input does not exist.
WARNING: Ignoring image left1_000055.png, because input does not exist.
WARNING: Ignoring image left1_000053.png, because input does not exist.
WARNING: Ignoring image left1_000051.png, because input does not exist.
WARNING: Ignoring image left1_000049.png, because input does not exist.
WARNING: Ignoring image left1_000047.png, because input does not exist.
WARNING: Ignoring image left1_000045.png, because input does not exist.
WARNING: Ignoring image left1_000043.png, because input does not exist.
WARNING: Ignoring image left1_000041.png, because input does not exist.
WARNING: Ignoring image left1_000039.png, because input does not exist.
WARNING: Ignoring image left1_000037.png, because input does not exist.
WARNING: Ignoring image left1_000035.png, because input does not exist.
WARNING: Ignoring image left1_000033.png, because input does not exist.
WARNING: Ignoring image left1_000031.png, because input does not exist.
WARNING: Ignoring image left1_000029.png, because input does not exist.
WARNING: Ignoring image left1_000027.png, because input does not exist.
WARNING: Ignoring image left1_000001.png, because input does not exist.
WARNING: Ignoring image left1_000003.png, because input does not exist.
WARNING: Ignoring image left1_000005.png, because input does not exist.
WARNING: Ignoring image left1_000007.png, because input does not exist.
WARNING: Ignoring image left1_000009.png, because input does not exist.
WARNING: Ignoring image left1_000011.png, because input does not exist.
WARNING: Ignoring image left1_000013.png, because input does not exist.
WARNING: Ignoring image left1_000015.png, because input does not exist.
WARNING: Ignoring image left1_000017.png, because input does not exist.
WARNING: Ignoring image left1_000019.png, because input does not exist.
WARNING: Ignoring image left1_000021.png, because input does not exist.
WARNING: Ignoring image left1_000023.png, because input does not exist.
WARNING: Ignoring image left1_000025.png, because input does not exist.
WARNING: Ignoring image left1_000059.png, because input does not exist.
WARNING: Ignoring image left1_000061.png, because input does not exist.
WARNING: Ignoring image left1_000063.png, because input does not exist.
WARNING: Ignoring image left1_000065.png, because input does not exist.
WARNING: Ignoring image left1_000067.png, because input does not exist.
WARNING: Ignoring image left1_000069.png, because input does not exist.
WARNING: Ignoring image left1_000071.png, because input does not exist.
WARNING: Ignoring image left1_000073.png, because input does not exist.
WARNING: Ignoring image left1_000075.png, because input does not exist.
WARNING: Ignoring image left1_000077.png, because input does not exist.
WARNING: Ignoring image left1_000079.png, because input does not exist.
WARNING: Ignoring image left1_000081.png, because input does not exist.
WARNING: Ignoring image left1_000083.png, because input does not exist.
WARNING: Ignoring image left1_000085.png, because input does not exist.
WARNING: Ignoring image left1_000087.png, because input does not exist.
WARNING: Ignoring image left1_000089.png, because input does not exist.
WARNING: Ignoring image left1_000091.png, because input does not exist.
WARNING: Ignoring image left1_000093.png, because input does not exist.
WARNING: Ignoring image left1_000095.png, because input does not exist.
WARNING: Ignoring image left1_000097.png, because input does not exist.
WARNING: Ignoring image left1_000099.png, because input does not exist.
WARNING: Ignoring image left1_000101.png, because input does not exist.
WARNING: Ignoring image left1_000103.png, because input does not exist.
WARNING: Ignoring image left1_000105.png, because input does not exist.
WARNING: Ignoring image left1_000107.png, because input does not exist.
WARNING: Ignoring image left1_000109.png, because input does not exist.
WARNING: Ignoring image left1_000111.png, because input does not exist.
WARNING: Ignoring image left1_000113.png, because input does not exist.
WARNING: Ignoring image left1_000115.png, because input does not exist.
WARNING: Ignoring image left1_000117.png, because input does not exist.
WARNING: Ignoring image right1_000058.png, because input does not exist.
WARNING: Ignoring image right1_000060.png, because input does not exist.
WARNING: Ignoring image right1_000062.png, because input does not exist.
WARNING: Ignoring image right1_000064.png, because input does not exist.
WARNING: Ignoring image right1_000066.png, because input does not exist.
WARNING: Ignoring image right1_000068.png, because input does not exist.
WARNING: Ignoring image right1_000070.png, because input does not exist.
WARNING: Ignoring image right1_000072.png, because input does not exist.
WARNING: Ignoring image right1_000074.png, because input does not exist.
WARNING: Ignoring image right1_000076.png, because input does not exist.
WARNING: Ignoring image right1_000078.png, because input does not exist.
WARNING: Ignoring image right1_000080.png, because input does not exist.
WARNING: Ignoring image right1_000082.png, because input does not exist.
WARNING: Ignoring image right1_000084.png, because input does not exist.
WARNING: Ignoring image right1_000086.png, because input does not exist.
WARNING: Ignoring image right1_000088.png, because input does not exist.
WARNING: Ignoring image right1_000090.png, because input does not exist.
WARNING: Ignoring image right1_000092.png, because input does not exist.
WARNING: Ignoring image right1_000094.png, because input does not exist.
WARNING: Ignoring image right1_000096.png, because input does not exist.
WARNING: Ignoring image right1_000098.png, because input does not exist.
WARNING: Ignoring image right1_000100.png, because input does not exist.
WARNING: Ignoring image right1_000102.png, because input does not exist.
WARNING: Ignoring image right1_000104.png, because input does not exist.
WARNING: Ignoring image right1_000106.png, because input does not exist.
WARNING: Ignoring image right1_000108.png, because input does not exist.
WARNING: Ignoring image right1_000110.png, because input does not exist.
WARNING: Ignoring image right1_000112.png, because input does not exist.
WARNING: Ignoring image right1_000114.png, because input does not exist.
WARNING: Ignoring image right1_000116.png, because input does not exist.
WARNING: Ignoring image right1_000118.png, because input does not exist.
WARNING: Ignoring image right1_000120.png, because input does not exist.
WARNING: Ignoring image right1_000122.png, because input does not exist.
WARNING: Ignoring image right1_000124.png, because input does not exist.
WARNING: Ignoring image right1_000126.png, because input does not exist.
WARNING: Ignoring image right1_000128.png, because input does not exist.
WARNING: Ignoring image right1_000130.png, because input does not exist.
WARNING: Ignoring image right1_000132.png, because input does not exist.
WARNING: Ignoring image right1_000134.png, because input does not exist.
WARNING: Ignoring image right1_000136.png, because input does not exist.
WARNING: Ignoring image right1_000138.png, because input does not exist.
WARNING: Ignoring image right1_000140.png, because input does not exist.
WARNING: Ignoring image right1_000142.png, because input does not exist.
WARNING: Ignoring image right1_000144.png, because input does not exist.
WARNING: Ignoring image right1_000146.png, because input does not exist.
WARNING: Ignoring image right1_000148.png, because input does not exist.
WARNING: Ignoring image right1_000150.png, because input does not exist.
WARNING: Ignoring image right1_000152.png, because input does not exist.
WARNING: Ignoring image right1_000154.png, because input does not exist.
WARNING: Ignoring image right1_000156.png, because input does not exist.
WARNING: Ignoring image right1_000158.png, because input does not exist.
WARNING: Ignoring image right1_000160.png, because input does not exist.
WARNING: Ignoring image right1_000162.png, because input does not exist.
WARNING: Ignoring image right1_000164.png, because input does not exist.
WARNING: Ignoring image right1_000166.png, because input does not exist.
WARNING: Ignoring image right1_000168.png, because input does not exist.
WARNING: Ignoring image right1_000170.png, because input does not exist.
WARNING: Ignoring image right1_000172.png, because input does not exist.
WARNING: Ignoring image right1_000174.png, because input does not exist.
WARNING: Ignoring image right1_000176.png, because input does not exist.
WARNING: Ignoring image right1_000178.png, because input does not exist.
WARNING: Ignoring image right1_000180.png, because input does not exist.
WARNING: Ignoring image right1_000182.png, because input does not exist.
WARNING: Ignoring image right1_000184.png, because input does not exist.
WARNING: Ignoring image right1_000186.png, because input does not exist.
WARNING: Ignoring image right1_000188.png, because input does not exist.
WARNING: Ignoring image right1_000190.png, because input does not exist.
WARNING: Ignoring image right1_000192.png, because input does not exist.
WARNING: Ignoring image right1_000194.png, because input does not exist.
WARNING: Ignoring image right1_000196.png, because input does not exist.
Fusing image [1/197] in 0.000s (0 points)
WARNING: Could not fuse any points. This is likely caused by incorrect settings - filtering must be enabled for the last call to patch match stereo.
Number of fused points: 0
Elapsed time: 0.012 [minutes]
Writing output: data/hypernerf/vrig/broom2/colmap/dense/workspace/fused.ply

Have I made any mistakes in the steps? I hope you can help me resolve this. Thank you very much.

guanjunwu commented 11 months ago

ERROR: Dense stereo reconstruction requires CUDA, which is not available on your system. I guess you should install the correct cuda toolkit?

kevinyu-0910 commented 11 months ago

My CUDA version is 11.8, although it's different from the version you used for running. However, I successfully ran it on the D-NeRF dataset. Does this mean it is feasible?

guanjunwu commented 11 months ago

Hi,I reproduced full process from another computer but it indeed can work I think there are no problem about your cuda or torch version. But can you use the colmap to calculate poses from any other dataset such as colmap? Here is a script to preprocess colmap format dataset:

workdir=“data/nsff/Balloon1” # or any other colmap format dataset
echo $workdir
mkdir -p $workdir/dense/workspace
colmap image_undistorter --image_path $workdir/images_colmap --input_path $workdir/sparse/0 --output_path $workdir/dense/workspace
colmap patch_match_stereo --workspace_path $workdir/dense/workspace --workspace_format COLMAP --PatchMatchStereo.geom_consistency true
# colmap stereo_fusion --workspace_path $workdir/dense/workspace --output_path $workdir/dense/workspace/fused.ply
mkjjang3598 commented 11 months ago

Hi, In my case, I've fixed export CUDA_VISIBLE_DEIVCES=1 to 0 https://github.com/hustvl/4DGaussians/issues/55

KevinYu0910 commented 10 months ago

I recently followed the guidance provided and adjusted the export CUDA_VISIBLE_DEVICES setting from 1 to 0. Unfortunately, I am still facing the same challenges as previously detailed. I would be most grateful for any further advice or insights you might offer to help resolve these issues.

kevinyu-0910 commented 10 months ago

I've identified the issue: it was a problem with my version of colmap. I have now updated to the latest version and it's working successfully. Thank you very much for your help.