Inconsistency between the view used in the SDF loss term and the other loss terms

[voyleg]

Hi, thank you for your great work!

I was testing Geo-Neus on my own data and noticed that, on each iteration of training, you sample rays for a random image image_perm[self.iter_step % len(image_perm)] at exp_runner.py#L126, but sample guidance points for a generally different image self.iter_step % len(image_perm) at exp_runner.py#L145-148.

So, I wonder if this is a bug or a feature --- could you please comment on that?

[songsenIng]

Hello, I would like to ask how to obtain the pairs.txt file and the points.npy and view_id.npy files in the sfm_pts folder for a custom data set? Looking forward to your reply, thank you！

[voyleg]

Hi, @songsenIng!

I used the dense reconstruction from COLMAP for guidance instead of the sparse reconstruction as the authors originally did. So, I describe how to get the files from the dense reconstruction outputs, but the procedure for the sparse reconstruction is similar.

To get points.npy, you need to

1. Read the point cloud reconstruction produced by COLMAP in PLY / TXT / BIN format,
2. Transform the points to NeuS space using scale_mat_inv from cameras.npz,
3. Save the points as a N x 3 numpy array into npy.

To get pairs.txt you can

1. Compute view selection scores following MVSNet, specifically with this part of the script,
2. Transform the pair.txt in MVSNet format to pairs.txt in Geo-Neus format, which is

   
   ref_view_i_1.{ext} src_view_i_1_1.{ext} src_view_i_1_2.{ext} ...
   ref_view_i_2.{ext} src_view_i_2_1.{ext} src_view_i_2_2.{ext} ...
   ...

where `{ext}` is the image file extension.

To get the `view_id.npy`, you can
1. Read the point visibility information from the COLMAP's `dense/sparse/images.bin` and `dense/fused.ply.vis` using the [COLMAP IO tools](https://github.com/colmap/colmap/blob/main/scripts/python),
2. And save it as a list of numpy arrays `[arr_0, arr_1, ..., arr_view_i, ...]` into npy. Each array contains the ids of points visible from the respective view.

In practice, it seems like you can ignore `view_id.npy` and just sample the guidance points randomly at each iteration ---
that is what the original implementation practically does anyway.

[kongsemancheng]

Hello, I would like to obtain thecamera.npz for my blendedMVS data. Could you please tell me how to get the camera.npz and normalize the coordinates to (-1,1)? I am looking forward to your reply, thank you very much.