search

SharkWipf / nerf_dataset_preprocessing_helper

Intelligent(?) image sharpness filtering for optimized NeRF training datasets
MIT License
62 stars 4 forks source link

How to modify 02_filter_colmap_data.py for original 3DGS colmap format data #2

Closed pknmax closed 3 months ago

pknmax commented 8 months ago

How to modify 02_filter_colmap_data.py for original 3DGS style input data as it is using colmap format data i.e. data includes folders like distorted, images (undistorted), sparse, stero. And not NeRFStudio's nsprocess generated json data?

SharkWipf commented 8 months ago

Unfortunately, I don't think the same concept really applies to 3DGS (neither Inria's nor Nerfstudio's Splatfacto). NeRFs (in Nerfstudio) only take the images and camera positions as input, as given by transforms.json. 3DGS, on the other hand, takes the images, positions (I think) and sparse point cloud as input. While it might be possible to modify the COLMAP generated database file directly to remove the least sharp images, it will still use the full point cloud as input.

This might not be a problem, in fact, it might be beneficial, but it's a lot more complex than removing a few lines from a json file as we currently do with transforms.json.

I might try to make it work with Nerfstudio's Splatfacto in the future, if I have time (no promises though), but I have little interest in making it work with Inria's version, as I don't use it, and Inria's license makes it a very unattractive solution compared to Nerfstudio. I'm open to contributions though.

I'm also looking into other methods that could completely remove the need for this project (and for COLMAP in general), but I don't know when, if ever, that will bear fruit yet.