search

facebookresearch / iSDF

Real-time Neural Signed Distance Fields for Robot Perception
https://joeaortiz.github.io/iSDF/
MIT License
417 stars 51 forks source link

open3d gui viewer with new train script #9

Closed joeaortiz closed 2 years ago

joeaortiz commented 2 years ago

Open3d gui viewer instead of trimesh and opencv viewers.

suddhu commented 2 years ago

Going to try this out, I forsee explicitly adding torch==1.11.0+cu113 to environment.yml to give users problems depending on their CUDA versions. Maybe remove version specification for torch, torchaudio, torchvision?

joeaortiz commented 2 years ago

Going to try this out, I forsee explicitly adding torch==1.11.0+cu113 to environment.yml to give users problems depending on their CUDA versions. Maybe remove version specification for torch, torchaudio, torchvision?

Thanks for spotting that. Installing torch should be done separately and not through the enivronment.yml so I'll remove them from there.

suddhu commented 2 years ago

Tried this out @joeaortiz and it really is excellent! Few initial thoughts:

joeaortiz commented 2 years ago

Tried this out @joeaortiz and it really is excellent! Few initial thoughts:

  • Would be great to have a screenshot button in the GUI (shouldn't be too hard?)
  • When you toggle pause, how does one refresh mesh visualization (for example, if I would like to color the mesh by normals with the checkbox)?
  • For python train_vis.py --config configs/replicaCAD.json I get this error at the end of sequences: ************************************** End of sequence ************************************** Exception in thread UpdateMain: Traceback (most recent call last): File "/home/robospare/anaconda3/envs/tac_neural/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/home/robospare/anaconda3/envs/tac_neural/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/home/robospare/suddhu/catkin_ws/src/tac_neural/isdf/visualisation/isdf_window.py", line 551, in update_main kf_im = o3d.geometry.Image(np.hstack(ims)) File "<__array_function__ internals>", line 5, in hstack File "/home/robospare/anaconda3/envs/tac_neural/lib/python3.8/site-packages/numpy/core/shape_base.py", line 346, in hstack return _nx.concatenate(arrs, 1) File "<__array_function__ internals>", line 5, in concatenate ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 3 dimension(s) and the array at index 1 has 2 dimension(s) ^CTraceback (most recent call last):

Thanks Suddhu, these are some good suggestions and the latter two I am actually already looking into. I'll make some changes and get these pushed in the next couple of days.