Problems running colmap_utils script

[jiadingfang]

Hi,

Thanks for the repo. I was trying to run SCNeRF with only images, but after looking at the code and issues related, it seems like I need to run colmap_utils script nontheless. But there are several errors trying to run the script:

File "/home/SCNeRF/colmap_utils/read_sparse_model.py", line 378, in main
    depth_ext = os.listdir(os.path.join(args.working_dir, "depth"))[0][-4:]

and

File "/home/SCNeRF/colmap_utils/post_colmap.py", line 33, in load_colmap_data
    with open(os.path.join(realdir, "train.txt"), "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/data/TUM_desk_rgb/train.txt'

I checked the code, I think the error happens because there is no depth output from colmap directly, and no idea what is train.txt. Could you double check the provided script work for a pure rgb dataset all the way through?

And if possible, it would be very helpful if you could provide a more detailed guide on how to run with only image inputs.

Thanks in advance!

[jeongyw12382]

First, I also suffered from similar problems where no depth outputs are acquired from COLMAP. In many cases, COLMAP failed to converge so that no outputs were provided. I strongly recommend you to use different descriptors, e.g., SuperPoint, D2Net, ... . If your program successfully returned the output depths but caused such problem, could you elaborate more about the issue?

[jeongyw12382]

Please let me know, if you have any problems of using raw images. I recommend giving me a mail. I could have a short online meeting for this. jeongyw12382@postech.ac.kr

[jiadingfang]

Thanks for the quick reply! I still have some questions regarding your answers, so maybe let me be more detailed about my question.

1. In terms of COLMAP outputs, in my experience, the sparse mode returns camera.bin, images.bin and points3D.bin (these can be output as txt if you so choose to), and the dense returns folders of consistency_graphs, depth_maps, normal_maps and fused ply. In your colmap_utils scripts, when you are asking for a depth folder (regarding the first error I posted), are you asking for some processed depths from the COLMAP output depth_maps folder?
2. Regarding my second error, which misses a train.txt file. This is not related to COLMAP outputs, are we supposed to create on our own as if we are creating a train split from our raw rgb inputs?

Overall, I expect your provided colmap_utils scripts would just work through for a raw image folder inputs, however it seems like there are still something extra we need to sort out. If this process could be straighten up, it would be very helpful because I observe a lot of interest using SCNeRF from only RGB inputs, which is probably the most common case in real life.

[jeongyw12382]

Hi. Probably I misunderstood your questions in the first reply. In addition, I've just received your mail. Thanks.

First, the data where I tested the script was Redwood dataset in my different private repository. So, you can ignore reorganizing the depth file. I'll soon add the code for this.

Second, If you are missing the "train.txt" files, then this is probably because of somewhat errors appeared before running the code line below. https://github.com/POSTECH-CVLab/SCNeRF/blob/1c32978f0c855ad06e7581640a0bfeb52973b3d3/colmap_utils/read_sparse_model.py#L411 Here are the checklists you should consider.

1. Make sure that your COLMAP does not produce multiple models. This may result in a poor convergence or insufficient number of estimated poses.
2. If any of COLMAP process including "colmap_utils.read_sparse_model" failed, the process will not properly run.

[jeongyw12382]

I have just pushed c473105ea0f3a54a9ce10c79cc65f06f6404da1a on the main branch. Could you retry the code using the most recent repository.

In my opinion, if "depth" issue was solved, the code would also work for the second issue.

[jeongyw12382]

Closing the issue since there were no replies for 2 days. Feel free to reopen the issue if the aforementioned commit hadn't work.