nerf_synthetic dataset support

[liujie-zheng]

Hi there! Does this approach support datasets such as nerf_synthetic with train, test and transforms.json?

[liujie-zheng]

Also for the Nerf On-the-go dataset, do I need to run COLMAP on them before training (since sparse is missing)?

[lilygoli]

The semantic data parser that we have included in the examples/datasets/colmap.py file is built on top of COLMAP parse. However it can easily be transferred to other type of parsers as it only adds semantic feature loading on top of the usual image and camera pose loading. The dataset format and naming should also follow the format described in #4. For all the datasets that we use in the paper we have run COLMAP on them first.

[mazy1998]

Hi im having the same issue when I run my own COLMAP on the Nerf On-the-go dataset, it doesn't work, can you kindly give the CLI commands for running COLMAP for directory including the files, "images"-"SD"-"split.json" (given that I have ran prep_data.py already)

Specifically I get this error.

Traceback (most recent call last):
  File "/home/mazeyar/SpotLessSplats/examples/spotless_trainer.py", line 1265, in <module>
    main(cfg)
  File "/home/mazeyar/SpotLessSplats/examples/spotless_trainer.py", line 1245, in main
    runner = Runner(cfg)
             ^^^^^^^^^^^
  File "/home/mazeyar/SpotLessSplats/examples/spotless_trainer.py", line 283, in __init__
    self.parser = SemanticParser(
                  ^^^^^^^^^^^^^^^
  File "/home/mazeyar/SpotLessSplats/examples/datasets/colmap.py", line 258, in __init__
    super().__init__(
  File "/home/mazeyar/SpotLessSplats/examples/datasets/colmap.py", line 228, in __init__
    K_undist, roi_undist = cv2.getOptimalNewCameraMatrix(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cv2.error: OpenCV(4.10.0) /io/opencv/modules/calib3d/src/undistort.dispatch.cpp:411: error: (-215:Assertion failed) CV_IS_MAT(_distCoeffs) && (_distCoeffs->rows == 1 || _distCoeffs->cols == 1) && (_distCoeffs->rows*_distCoeffs->cols == 4 || _distCoeffs->rows*_distCoeffs->cols == 5 || _distCoeffs->rows*_distCoeffs->cols == 8 || _distCoeffs->rows*_distCoeffs->cols == 12 || _distCoeffs->rows*_distCoeffs->cols == 14) in function 'cvUndistortPointsInternal'

[mazy1998]

changing this code.

if type_ == 2 or type_ == "SIMPLE_RADIAL": params = np.array([cam.k1, 0.0, 0.0, 0.0], dtype=np.float32) camtype = "perspective"

seems to help, I added extra 0s to params as described here.

https://amroamroamro.github.io/mexopencv/matlab/cv.getOptimalNewCameraMatrix.html