search

Anttwo / SuGaR

[CVPR 2024] Official PyTorch implementation of SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering
https://anttwo.github.io/sugar/
Other
2.31k stars 175 forks source link

RuntimeError: shape is invalid for input of size #149

Open rapon232 opened 8 months ago

rapon232 commented 8 months ago

I have done evetything correctly and i am getting this error! Why does the input shape not match my image dimensions? Why is it initializing randomly (it changes every time I run it)?

Traceback (most recent call last): File "/content/SuGaR/train.py", line 127, in coarse_sugar_path = coarse_training_with_density_regularization(coarse_args) File "/content/SuGaR/sugar_trainers/coarse_density.py", line 526, in coarse_training_with_density_regularization gt_rgb = gt_image.view(-1, sugar.image_height, sugar.image_width, 3) RuntimeError: shape '[-1, 1024, 1920, 3]' is invalid for input of size 6059520

ywaison commented 7 months ago

I have done evetything correctly and i am getting this error! Why does the input shape not match my image dimensions? Why is it initializing randomly (it changes every time I run it)?

Traceback (most recent call last): File "/content/SuGaR/train.py", line 127, in coarse_sugar_path = coarse_training_with_density_regularization(coarse_args) File "/content/SuGaR/sugar_trainers/coarse_density.py", line 526, in coarse_training_with_density_regularization gt_rgb = gt_image.view(-1, sugar.image_height, sugar.image_width, 3) RuntimeError: shape '[-1, 1024, 1920, 3]' is invalid for input of size 6059520

I think the resolution of the images you input is not all the same, so this is the cause of the problem

GoroYeh-HRI commented 4 months ago

I have done evetything correctly and i am getting this error! Why does the input shape not match my image dimensions? Why is it initializing randomly (it changes every time I run it)?

Traceback (most recent call last): File "/content/SuGaR/train.py", line 127, in coarse_sugar_path = coarse_training_with_density_regularization(coarse_args) File "/content/SuGaR/sugar_trainers/coarse_density.py", line 526, in coarse_training_with_density_regularization gt_rgb = gt_image.view(-1, sugar.image_height, sugar.image_width, 3) RuntimeError: shape '[-1, 1024, 1920, 3]' is invalid for input of size 6059520

I got exactly the same errors. @ywaison I used the same set of input/images to run this script. I believe the resolution of images are the same. Is there any possible reasons I'm encountering this error? Thank you very much.

GoroYeh-HRI commented 4 months ago

All my input images are size 1280 x 720 image

However, when I checked the cameras.json output from gaussian_splatting/train.py, the resolutions are various and not the same as my image size.

image

I think this might be the reason why there would be a shape mismatch when training the SuGaR model. Do you know why this happened? Thank you very much.

projan-shakya commented 3 months ago

@GoroYeh-HRI are You using hierarchical colmap mapper? I have the same issue going on rn. Did you fix it?

misakamage commented 3 months ago

I have the same issue

michelgokan commented 3 weeks ago

Same issue here. Did you find any solution?

psemalah commented 1 week ago

@GoroYeh-HRI are You using hierarchical colmap mapper? I have the same issue going on rn. Did you fix it?

Same issue. I also use the hierarchical colmap mapper. Can I please ask did you solve it ? Many thanks.

michelgokan commented 1 week ago

@psemalah In my case, the issue was that I was using the wrong set of images—the modified ones. Once I switched to the original set, everything started to work.

GoroYeh-HRI commented 1 week ago

I was using the older README instruction. I built my COLMAP model separately using point_triangulator with know camera poses. These are my steps to train SuGaR:

SuGaR steps:

Assuming you have a reconstruction built from COLMAP: model/ containing following files:

Let's say you want to train the SuGaR model in ~/target_folder and you clone this repository in ~/SuGaR/.

  1. Copy you images to target_folder/input/

  2. Copy the reconstruction to target_folder/distorted/sparse/0/

  3. Run python gaussian_splatting/convert.py --skip_matching 

    python gaussian_splatting/convert.py -s ../target_folder/ --skip_matching

  4. Use cloud_mapper read_write_model.py to convert target_folder/distorted/sparse/0/ from bin to txt format

    cd ~/colmap/scripts/python
python read_write_model.py --input_model ~/target_folder/distorted/sparse/0/ --output_model ~/target_folder/distorted/sparse/0/ --output_format .txt

  5. Modify cameras.txt: use original triangulation input cameras.txt. Change FULL_OPENCV to PINHOLE. Delete distortion parameters

  6. Modify target_folder/images/: copy and paste target_foler/input/ images.

  7. Train the GS model:

    python gaussian_splatting/train.py -s ../target_folder/ --iterations 7000 -m ../target_folder-GS

  8. Train the SuGaR model:

    python train.py  -s ../target_folder -c ../target_folder-GS/ -r "sdf"
psemalah commented 1 week ago

@michelgokan @GoroYeh-HRI Thanks so much for your help. The problem is the caused by the resize of the image, operated by the undistotion function of colmap implemented in the convert.py.

I found a more simple way to make the size consistent after to the colmap distortion:

just simply run the following instead of the convert.py

colmap image_undistorter \ --image_path $DATASET_PATH/images \ --input_path $DATASET_PATH/sparse/0 \ --output_path $DATASET_PATH/dense \ --output_type COLMAP \ --min_scale 1 --max_scale 1

which make make your image add black blank on the edge but will not significantly affect your results.