KevinXu02 / splatfacto-w

Apache License 2.0
73 stars 2 forks source link

self.num_points suddenly decreased to 0 when training on my own dataset. #12

Open xuxumiao777 opened 4 weeks ago

xuxumiao777 commented 4 weeks ago

Hi, author! Thanks for your fantastic work. I have encountered a problem when training on my own datasets. I have created tsv file for my own dataset. But the training process was terminated because self.num_points and num_splits suddenly decreased to zero(yes, I have monitered this parameter during the whole process). All datasets creashed at about 3000 iterations. image

hope for your answer.

KevinXu02 commented 4 weeks ago

Can you please try using splatfacto-w-light? That method should be using nerfstudio dataparser and might work for such datasets. splatfacto-w was basically designed for phototourism datasets.

xuxumiao777 commented 4 weeks ago

Can you please try using splatfacto-w-light? That method should be using nerfstudio dataparser and might work for such datasets. splatfacto-w was basically designed for phototourism datasets.

I have tried splatfacto-w-light, but it needs transforms.json. I have used colmap2nerf.py from nerf to generate a transforms.json. Honestly, it can run fluently without any error, but the result is a mess. I guess the coordinate system is not correct. Could you please provide a script, so that i can generate a correct transforms.json from colmap. Thanks.

KevinXu02 commented 4 weeks ago

ns-train splatfacto-w-light [OPTIONS] --data [PATH] [dataparser] colmap You can use colmap dataparser this way. And I think if you add --help in the end it will show the usage of colmap datapraser from nerfstudio.

xuxumiao777 commented 4 weeks ago

ns-train splatfacto-w-light [OPTIONS] --data [PATH] [dataparser] colmap You can use colmap dataparser this way. And I think if you add --help in the end it will show the usage of colmap datapraser from nerfstudio.

image tried_but_failed😂, here is my command: ns-train splatfacto-w --data /gpfs/dataset/jingxigudao_tiankong/VID_20240804_111824/undistorted_colmap/dense --pipeline.datamanager.eval-image-indices None --viewer.make-share-url True colmap

KevinXu02 commented 4 weeks ago

Please try splatfacto-w-light.

xuxumiao777 commented 4 weeks ago

Please try splatfacto-w-light.

Sorry for that, I have now used splatfacto-w-light. It can run smoothly. But I can only see a mess in the viewer, which is different from the result by command splatfacto-w. image

KevinXu02 commented 4 weeks ago

Please consider enabling these options.


If you want to train datasets without nerf-w's train/test split or your own datasets, we provided a light-weight version of the method for general cases. To train with it, you can run the following command

ns-train splatfacto-w-light [OPTIONS] --data [PATH] [dataparser]
You can add these options to enhance the training process:

--pipeline.model.enable_bg_model True

Enables background modeling
Useful for scenes with distinct foreground and background
--pipeline.model.enable_alpha_loss True

Enables alpha loss for punishing gaussians from occupying sky areas.
Should be used with background modeling
--pipeline.model.enable_robust_mask True

Improves handling of transient objects (things that appear/disappear between views)
xuxumiao777 commented 4 weeks ago

Please consider enabling these options.

If you want to train datasets without nerf-w's train/test split or your own datasets, we provided a light-weight version of the method for general cases. To train with it, you can run the following command

ns-train splatfacto-w-light [OPTIONS] --data [PATH] [dataparser]
You can add these options to enhance the training process:

--pipeline.model.enable_bg_model True

Enables background modeling
Useful for scenes with distinct foreground and background
--pipeline.model.enable_alpha_loss True

Enables alpha loss for punishing gaussians from occupying sky areas.
Should be used with background modeling
--pipeline.model.enable_robust_mask True

Improves handling of transient objects (things that appear/disappear between views)

use the following command and still a mess ns-train splatfacto-w-light --data /gpfs/dataset/jingxigudao_tiankong/VID_20240804_111824/undistorted_colmap/dense --pipeline.datamanager.eval-image-indices None --viewer.make-share-url True --pipeline.model.enable_bg_model True --pipeline.model.enable_alpha_loss True --pipeline.model.enable_robust_mask True colmap strange