search

cvachha / instruct-gs2gs

Gaussian splatting implementation of Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions
https://instruct-gs2gs.github.io/
MIT License
62 stars 1 forks source link

Does not recognize colmap dataset #6

Open StephenYangjz opened 7 months ago

StephenYangjz commented 7 months ago

Hi, it seems like when I try to do the editing on a colmap dataset with a flag to load a model checkpoint, it doesnt recognize the arguments. How may I go around this? Thank you! image

ayaanzhaque commented 6 months ago

remove the word colmap and it will work

StephenYangjz commented 6 months ago

Hi @ayaanzhaque , I did try but then it's not able to recognize the dataset as a colmap dataset, and will look for transforms.json file, which I do not have locally.

ayaanzhaque commented 6 months ago

what command did you use to train the base GS?

StephenYangjz commented 6 months ago

I used sth like ns-train splatfacto colmap --data path/to/data @ayaanzhaque

ayaanzhaque commented 6 months ago

oh maybe this is because we built our code before the official splatfacto release. @cvachha any ideas here?

cvachha commented 6 months ago

Hi @StephenYangjz, can you try re-training the dataset with the splatfacto command without writing colmap: ns-train splatfacto --data path/to/data Then run igs2gs also without writing colmap. I assume you are trying the instruct-nerf2nerf bear dataset, and I am able to get it to run with igs2gs without writing colmap: ns-train igs2gs --data path/to/data --load-dir path\to\nerfstudio_models --pipeline.prompt "make the bear a polar bear" Also the load-dir should be to the nerfstudio_models folder.

Let me know if this works!

StephenYangjz commented 6 months ago

Hi @cvachha thanks for the reply. I can certainly run it that way but then what's the point of me running colmap? I did so to get the better gaussian initilizations via their pointcloud, so it will not make sense if I dont use those info...

cvachha commented 6 months ago

Hi, I think that the original bear dataset was originally captured with polycam instead of colmap, so I believe by default splatfacto will run from a random point clound initialization. You could try copying over the images of the dataset into a new folder and process that with ns-process-data which should be running colmap, and then run splatfacto, which I believe would use the colmap initialization. There is some info regarding point cloud initialization here