npz normal files on private data

[pablovela5620]

Hey, congrats on the release! I've been trying to run the data preprocessing script and manage to get the majority of it working. The only missing component is that using SNU generates .png files but not the .npz files that the reconstruction requires. Can you give some insight as to how the .npz files were generated and if the should be normalized/mean-centered?

[pablovela5620]

I attempt using the following to generate the .npz files on line 114 of exp_preprocess.py

og_results_path = Path(dir_neuris) / 'image' / 'results'
og_pred_norm_path = Path(dir_neuris) / 'pred_normal'

norm_images = sorted(list(og_results_path.glob("*pred_norm.png")))
for norm_path in norm_images:
    norm_img = cv2.imread(str(norm_path))
    norm_ = ((norm_img)/(255*.5)) - 1
    pred_norm_file = og_pred_norm_path / norm_path.stem[:4]
    cv2.imwrite(str(og_pred_norm_path / f'{norm_path.stem[:4]}.png'), norm_img)
    np.savez_compressed(pred_norm_file, norm_)

but training leads to a bunch of noise

[jiepengwang]

I uploaded our test.py file. You can replace the test.py in SNU with this file https://connecthkuhk-my.sharepoint.com/:u:/g/personal/jiepeng_connect_hku_hk/EesBqZWh3exJjOT9tcdflFkBPI1TP1T1ly9xAozvxC0RUQ?e=drdfqD. Please try again.

[pablovela5620]

great thank you for that! I got it working and training. Now the issue I face is that it seems to be producing bad outputs. As an example here is what the custom iphone data outputs for me on the first 2000 iterations

on my custom data, after following the provided preprocessing I get the following

Do you have any recommendations as to where I could try to look and where things could be failing? Thanks. It could also be the case that I just need to wait longer, but it seems like the original had at least some level of convergence in the first 2k iterations

[LiXinghui-666]

great thank you for that! I got it working and training. Now the issue I face is that it seems to be producing bad outputs. As an example here is what the custom iphone data outputs for me on the first 2000 iterations

on my custom data, after following the provided preprocessing I get the following

Do you have any recommendations as to where I could try to look and where things could be failing? Thanks. It could also be the case that I just need to wait longer, but it seems like the original had at least some level of convergence in the first 2k iterations

Hey, I'd like to ask if your training has improved? I use the data collected by iphone to obtain the camera pose through colmap, and then after neuris processing, the final result is poor, I do not know what the problem is.

[jiepengwang]

Please make sure the calibrated camera poses are correct and normalized. And for your own data, it's better to capture the data with a proper angle and distance. When the camera observes the scene very locally as illustrated by the chessboard image, it's ambiguous to reason the geometry. Besides, one minor suggestion is to check the NeuS results and the normal predictions to find some clues. In general, NeuS can generate a rough geometry and the normal predictions should look reasonable. Hope this can provide some help.

[pablovela5620]

Appreciate the help! Turns out part of it was a mistake on my side where I didn't correctly initialize the image size in the preprocessing step. I'm using the chess scene from the 7scenes dataset and it seemed to work okay. Thank you for the tips!