Run custom / Global refine fails

tomole444 commented 1 year ago

After the first step completes successfully the second step (which is started automatically) fails. It also fails, when I try to run it manually. The console error is the following:

project train_images 161/272
project train_images 162/272
project train_images 163/272
project train_images 164/272
project train_images 165/272
project train_images 166/272
project train_images 167/272
Traceback (most recent call last):
  File "run_custom.py", line 247, in <module>
    run_one_video_global_nerf(out_folder=args.out_folder)
  File "run_custom.py", line 152, in run_one_video_global_nerf
    tracker.run_global_nerf(reader=reader, get_texture=True, tex_res=512)
  File "/home/grass/Documents/Leyh/BundleSDF/bundlesdf.py", line 773, in run_global_nerf
    mesh = nerf.mesh_texture_from_train_images(mesh, rgbs_raw=rgbs_raw, train_texture=False, tex_res=tex_res)
  File "/home/grass/Documents/Leyh/BundleSDF/nerf_runner.py", line 1511, in mesh_texture_from_train_images
    locations, distance, index_tri = trimesh.proximity.closest_point(mesh, pts)
  File "/opt/conda/envs/py38/lib/python3.8/site-packages/trimesh/proximity.py", line 152, in closest_point
    all_candidates = np.concatenate(candidates)
  File "<__array_function__ internals>", line 200, in concatenate
ValueError: need at least one array to concatenate

Your can find the full log here: globalRefine.log

My dataset is here: https://cloud.thws.de/s/bR2asfPPBjRHrmG

wenbowen123 commented 1 year ago

From the log, it seems the 167th or 168th image (among the 272 keyframes) is broken, can you check that and its segmentation?

tomole444 commented 12 months ago

These are the two masks for those two keyframes you mentioned.

00512 00515

Seems like the masks are not ideal.

wenbowen123 commented 12 months ago

Indeed the mask looks noisy. But I guess it's not the root cause.

File "/home/grass/Documents/Leyh/BundleSDF/nerf_runner.py", line 1511, in mesh_texture_from_train_images locations, distance, index_tri = trimesh.proximity.closest_point(mesh, pts)

From the log, can you print the shape of pts?

tomole444 commented 12 months ago

Alright. Seems like pts is empty: BookSmallRefine.log

tomole444 commented 11 months ago

Here you can find the output-data: https://cloud.thws.de/s/qWWK4CLKKZGMm3E

wenbowen123 commented 11 months ago

Pushed a fix. Can you try again?

NVlabs / BundleSDF

Run custom / Global refine fails #111