I have my dataset in the format that 3D Gaussian Splatting and NerfStudio accept. That is a folder with an images subfolder and camera params of those images saved in a json file. How do I change them in the format that PixelSplat accepts?

Shahid1Malik commented 2 months ago

i have my dataset in the format that 3D Gaussian Splatting and NerfStudio accepts. That is an image folder with camera parameters of those images saved in a json file. How do I change them in the format that PixelSplat accepts?

donydchen commented 2 months ago

Hi @Shahid1Malik, sorry for the late reply, I've been busy with several deadlines previously.

To run on your own dataset, the straightforward way is to convert your data into similar torch files so that you can easily reuse the data loader.

To convert data to torch files, refer to how we converted the DTU dataset at https://github.com/donydchen/mvsplat/blob/main/src/scripts/convert_dtu.py. We have also had more discussions regarding this convert script. Please refer to https://github.com/donydchen/mvsplat/issues/28.

If the camera parameters are in the format that 3D Gaussian Splatting and NerfStudio accept, my best guess is that they are obtained via COLMAP. If so, you can refer to https://github.com/donydchen/mvsplat/issues/1 for instructions on converting the COLMAP data.

After successfully building the torch files, the most important thing is to confirm that the camera parameters are correctly formatted. You can find more related discussions at https://github.com/donydchen/mvsplat/issues/23#issuecomment-2085190160.

Shahid1Malik commented 2 months ago

i have converted a subset of real10k using your convert.py script. However the torch file generated is 47 MB. when i feed it for evaluation, the testing doesnt work, it appears like it does not find or accept the torch file. What could be the problem here

donydchen commented 2 months ago

Hi @Shahid1Malik, several reasons can lead to these issues. Below, I provide some potential solutions to help you debug.

make sure the torch data is correctly loaded; you can do it by printing the self.chunks right after https://github.com/donydchen/mvsplat/blob/main/src/dataset/dataset_re10k.py#L83
if you are running the testing, make sure that the scenes exist in the eval index file (e.g., assets/evaluation_index_re10k.json); you can confirm it at https://github.com/donydchen/mvsplat/blob/main/src/dataset/view_sampler/view_sampler_evaluation.py#L54

donydchen / mvsplat

I have my dataset in the format that 3D Gaussian Splatting and NerfStudio accept. That is a folder with an images subfolder and camera params of those images saved in a json file. How do I change them in the format that PixelSplat accepts? #55