search

facebookresearch / 3D-Vision-and-Touch

When told to understand the shape of a new object, the most instinctual approach is to pick it up and inspect it with your hand and eyes in tandem. Here, touch provides high fidelity localized information while vision provides complementary global context. However, in 3D shape reconstruction, the complementary fusion of visual and haptic modalities remains largely unexplored. In this paper, we study this problem and present an effective chart-based approach to fusing vision and touch, which leverages advances in graph convolutional networks. To do so, we introduce a dataset of simulated touch and vision signals from the interaction between a robotic hand and a large array of 3D objects. Our results show that (1) leveraging both vision and touch signals consistently improves single-modality baselines, especially when the object is occluded by the hand touching it; (2) our approach outperforms alternative modality fusion methods and strongly benefits from the proposed chart-based structure; (3) reconstruction quality boosts with the number of grasps provided; and (4) the touch information not only enhances the reconstruction at the touch site but also extrapolates to its local neighborhood.
Other
68 stars 14 forks source link

Import #4

Closed SaiSakhamuri closed 2 years ago

SaiSakhamuri commented 2 years ago

import models import utils import data_loaders

I am getting below error in recon.py. Import "models" could not be resolvedPyright(reportMissingImports) Could you please help me with this error

EdwardSmith1884 commented 2 years ago

Hi, I am working on double checking everything and will push a new version of the code tomorrow which can work with more recent versions of pytorch and cuda. In the mean time can you let me know what environment you are using?

SaiSakhamuri commented 2 years ago

Hi,

I am using google collab environment. The steps I followed are as below.

  1. I have cloned using !git clone https://github.com/facebookresearch/3D-Vision-and-Touch.git
  2. pip install torch==1.4.0
  3. import os os.chdir('/content/3D-Vision-and-Touch')
  4. !bash download_data.sh
  5. !cd touch_charts
  6. !python touch_charts/recon.py --exp_type 1 --exp_id 1 after this step I am getting below error. Traceback (most recent call last): File "touch_charts/recon.py", line 18, in import utils ModuleNotFoundError: No module named 'utils' All the import statements of .py files in other files also showing errors. Thanks, Sai

On Thu, 7 Apr 2022 at 4:40 PM, Edward Smith @.***> wrote:

Hi, I am working on double checking everything and will push a new version of the code tomorrow which can work with more recent versions of pytorch and cuda. In the mean time can you let me know what environment you are using?

— Reply to this email directly, view it on GitHub https://github.com/facebookresearch/3D-Vision-and-Touch/issues/4#issuecomment-1092227977, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWHJNKEAN5PXJKJWCCFFPRDVD5I5JANCNFSM5S2PQIAA . You are receiving this because you authored the thread.Message ID: @.***>

EdwardSmith1884 commented 2 years ago

Hi, so I updated the code to use pytorch3D , meaning you can use much more recent version of pytorch and cuda. I would recommend pulling the new code, following the install instructions from scratch and seeing if this solves the issue. If not let me know and I will see what I can do.

SaiSakhamuri commented 2 years ago

Still getting same issues. When there are .py modules imported in other .py models it shows errors. Not sure, Whether I am missing anything to set the path. Please find the attached screenshots. When I execute the command !python recon.py --exp_type test --exp_id 1

Traceback (most recent call last): File "recon.py", line 18, in import utils File "../utils.py", line 11, in from pytorch3d.loss im ![Uploading Screen Shot 2022-04-10 at 1.59.35 PM.png…]() port chamfer_distance as cuda_cd ModuleNotFoundError: No module named 'pytorch3d' <img width="1440" alt="Screen Shot 2022-04-10 at 1 59 35 PM" src="https://user-images.githubusercontent.com/93230760/162635540-3764a6bc-56dc-4d1

Screen Shot 2022-04-10 at 1 59 53 PM Screen Shot 2022-04-10 at 1 59 17 PM

9-af73-c495139288e3.png">

Screen Shot 2022-04-10 at 1 59 35 PM
EdwardSmith1884 commented 2 years ago

Hi, so it looks like the path issue has been resolved as it reaches utils.py . Now it looks like you have not install pytorch3D. Try installing it and see if this resolves the problem.