search

JiehongLin / SAM-6D

[CVPR2024] Code for "SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation".
253 stars 19 forks source link

Fails when building PointNet2 (CUDA error) #32

Open matteomastrogiuseppe opened 2 months ago

matteomastrogiuseppe commented 2 months ago

Hey, Thanks a lot for sharing the source code.

I have problems when running the prepare.sh file, mostly related to the CUDA version. Here is the main error:

/home/matteomastro/miniforge3/envs/sam6d/lib/python3.9/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.7). Most likely this shouldn't be a problem.
  warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
building 'pointnet2._ext' extension

and after that I get a bunch of ninja-build error (missing header files and stuff). The problem seems to be related only to PointNet2, the rest of the build goes smooth. When running the demo.sh I get errors because PointNet is missing.

I'm running on Ubuntu 22.04, nvcc --version returns:

Cuda compilation tools, release 11.5, V11.5.119

while, nvidia-smi returns:

...
 NVIDIA-SMI 545.30                 Driver Version: 546.09       CUDA Version: 12.3
...

Do you have any idea what could be wrong? I get the same problem on multiple machines. Let me know if you need the full error log. Thanks for your help!

savidini commented 2 months ago

Hi @matteomastrogiuseppe,

maybe I can help you as I had a similar problem when trying to run inference. I don't think the difference between CUDA 12.3 on your system and 11.5 in the conda env should be a problem (I get the same warning).

After some trial and error, I can run inference on different systems with different package versions in my environment (see below). You can also try using my fork https://github.com/savidini/SAM-6D. In this case, please read the Getting Started as there are some changes. I have tested this on several cloud servers and my local machine and can run inference on the example and custom objects without errors.

Please let me know if this works for you, I might open a pull request in that case. If you still get errors, you can post a full error log.

Kind regards

Modified environment.yml ```yml name: sam6d channels: - xformers - conda-forge - pytorch - nvidia - defaults dependencies: - pip - python=3.9.6 - cuda=12.0 - libxcrypt - pip: - torch==2.2.1 - torchvision - fvcore - xformers - torchmetrics - blenderproc==2.7.0 - opencv-python # ISM - omegaconf - ruamel.yaml - hydra-colorlog - hydra-core - gdown - pandas - imageio - pyrender - pytorch-lightning==1.8.1 - pycocotools - distinctipy - git+https://github.com/facebookresearch/segment-anything.git # SAM - ultralytics==8.0.135 # FastSAM # PEM - timm - gorilla-core==0.2.7.8 - trimesh==4.0.8 - gpustat==1.0.0 - imgaug - einops ```
matteomastrogiuseppe commented 1 month ago

Hi @savidini,

Many thanks for reaching out. Your fork works! I also managed to run sam6d on a custom object smoothly. A PR would be great I think.

My working environment (from your yaml): ``` # packages in environment at /home/matteomastro/miniforge3/envs/sam6d: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge _sysroot_linux-64_curr_repodata_hack 3 h69a702a_14 conda-forge absl-py 2.1.0 pypi_0 pypi addict 2.4.0 pypi_0 pypi aiohttp 3.9.3 pypi_0 pypi aiosignal 1.3.1 pypi_0 pypi antlr4-python3-runtime 4.9.3 pypi_0 pypi asttokens 2.4.1 pyhd8ed1ab_0 conda-forge async-timeout 4.0.3 pypi_0 pypi attrs 23.2.0 pypi_0 pypi beautifulsoup4 4.12.3 pypi_0 pypi binutils 2.40 hdd6e379_0 conda-forge binutils_impl_linux-64 2.40 hf600244_0 conda-forge binutils_linux-64 2.40 hdade7a5_3 conda-forge blenderproc 2.7.0 pypi_0 pypi blessed 1.20.0 pypi_0 pypi bzip2 1.0.8 hd590300_5 conda-forge c-compiler 1.7.0 hd590300_0 conda-forge ca-certificates 2024.2.2 hbcca054_0 conda-forge certifi 2024.2.2 pypi_0 pypi charset-normalizer 3.3.2 pypi_0 pypi colorlog 6.8.2 pypi_0 pypi comm 0.2.2 pyhd8ed1ab_0 conda-forge contourpy 1.2.1 pypi_0 pypi cuda 12.0.0 ha804496_1 conda-forge cuda-cccl 12.0.90 ha770c72_1 conda-forge cuda-cccl-impl 2.0.1 ha770c72_1 conda-forge cuda-cccl_linux-64 12.0.90 ha770c72_1 conda-forge cuda-command-line-tools 12.0.0 ha770c72_0 conda-forge cuda-compiler 12.0.0 h6459364_1 conda-forge cuda-cudart 12.0.107 hd3aeb46_8 conda-forge cuda-cudart-dev 12.0.107 hd3aeb46_8 conda-forge cuda-cudart-dev_linux-64 12.0.107 h59595ed_8 conda-forge cuda-cudart-static 12.0.107 hd3aeb46_8 conda-forge cuda-cudart-static_linux-64 12.0.107 h59595ed_8 conda-forge cuda-cudart_linux-64 12.0.107 h59595ed_8 conda-forge cuda-cuobjdump 12.0.76 h59595ed_0 conda-forge cuda-cupti 12.0.90 h59595ed_2 conda-forge cuda-cupti-dev 12.0.90 h59595ed_2 conda-forge cuda-cuxxfilt 12.0.76 h59595ed_0 conda-forge cuda-driver-dev 12.0.107 hd3aeb46_8 conda-forge cuda-driver-dev_linux-64 12.0.107 h59595ed_8 conda-forge cuda-gdb 12.0.90 hd47b8d6_0 conda-forge cuda-libraries 12.0.0 ha770c72_1 conda-forge cuda-libraries-dev 12.0.0 ha770c72_1 conda-forge cuda-nsight 12.0.78 ha770c72_0 conda-forge cuda-nvcc 12.0.76 hba56722_12 conda-forge cuda-nvcc-dev_linux-64 12.0.76 ha770c72_1 conda-forge cuda-nvcc-impl 12.0.76 h59595ed_1 conda-forge cuda-nvcc-tools 12.0.76 h59595ed_1 conda-forge cuda-nvcc_linux-64 12.0.76 hba56722_12 conda-forge cuda-nvdisasm 12.0.76 h59595ed_0 conda-forge cuda-nvml-dev 12.0.76 hcb278e6_0 conda-forge cuda-nvprof 12.0.90 h59595ed_1 conda-forge cuda-nvprune 12.0.76 h59595ed_0 conda-forge cuda-nvrtc 12.0.76 hd3aeb46_2 conda-forge cuda-nvrtc-dev 12.0.76 hd3aeb46_2 conda-forge cuda-nvtx 12.0.76 h59595ed_1 conda-forge cuda-nvvp 12.0.90 h59595ed_0 conda-forge cuda-opencl 12.0.76 h59595ed_0 conda-forge cuda-opencl-dev 12.0.76 ha770c72_0 conda-forge cuda-profiler-api 12.0.76 ha770c72_0 conda-forge cuda-runtime 12.0.0 hd8ed1ab_2 conda-forge cuda-sanitizer-api 12.0.90 h59595ed_0 conda-forge cuda-toolkit 12.0.0 ha804496_1 conda-forge cuda-tools 12.0.0 ha770c72_0 conda-forge cuda-version 12.0 hffde075_3 conda-forge cuda-visual-tools 12.0.0 ha770c72_0 conda-forge cxx-compiler 1.7.0 h00ab1b0_0 conda-forge cycler 0.12.1 pypi_0 pypi debugpy 1.8.1 py39h3d6467e_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge distinctipy 1.3.4 pypi_0 pypi einops 0.7.0 pypi_0 pypi exceptiongroup 1.2.0 pyhd8ed1ab_2 conda-forge executing 2.0.1 pyhd8ed1ab_0 conda-forge filelock 3.13.3 pypi_0 pypi fire 0.6.0 pypi_0 pypi fonttools 4.50.0 pypi_0 pypi freetype-py 2.4.0 pypi_0 pypi frozenlist 1.4.1 pypi_0 pypi fsspec 2024.3.1 pypi_0 pypi fvcore 0.1.5.post20221221 pypi_0 pypi gcc 12.3.0 h95e488c_3 conda-forge gcc_impl_linux-64 12.3.0 he2b93b0_5 conda-forge gcc_linux-64 12.3.0 h6477408_3 conda-forge gdown 5.1.0 pypi_0 pypi gds-tools 1.5.0.59 hd3aeb46_1 conda-forge gmp 6.3.0 h59595ed_1 conda-forge gorilla-core 0.2.7.8 pypi_0 pypi gpustat 1.0.0 pypi_0 pypi grpcio 1.62.1 pypi_0 pypi gxx 12.3.0 h95e488c_3 conda-forge gxx_impl_linux-64 12.3.0 he2b93b0_5 conda-forge gxx_linux-64 12.3.0 h4a1b8e8_3 conda-forge h5py 3.10.0 pypi_0 pypi huggingface-hub 0.22.2 pypi_0 pypi hydra-colorlog 1.2.0 pypi_0 pypi hydra-core 1.3.2 pypi_0 pypi idna 3.6 pypi_0 pypi imageio 2.34.0 pypi_0 pypi imgaug 0.4.0 pypi_0 pypi importlib-metadata 7.1.0 pyha770c72_0 conda-forge importlib-resources 6.4.0 pypi_0 pypi importlib_metadata 7.1.0 hd8ed1ab_0 conda-forge iniconfig 2.0.0 pypi_0 pypi iopath 0.1.10 pypi_0 pypi ipykernel 6.29.3 pyhd33586a_0 conda-forge ipython 8.18.1 pyh707e725_3 conda-forge jedi 0.19.1 pyhd8ed1ab_0 conda-forge jinja2 3.1.3 pypi_0 pypi jupyter_client 8.6.1 pyhd8ed1ab_0 conda-forge jupyter_core 5.7.2 py39hf3d152e_0 conda-forge kernel-headers_linux-64 3.10.0 h4a8ded7_14 conda-forge kiwisolver 1.4.5 pypi_0 pypi lazy-loader 0.3 pypi_0 pypi ld_impl_linux-64 2.40 h41732ed_0 conda-forge libcublas 12.0.1.189 hd3aeb46_3 conda-forge libcublas-dev 12.0.1.189 hd3aeb46_3 conda-forge libcufft 11.0.0.21 hd3aeb46_2 conda-forge libcufft-dev 11.0.0.21 hd3aeb46_2 conda-forge libcufile 1.5.0.59 hd3aeb46_1 conda-forge libcufile-dev 1.5.0.59 hd3aeb46_1 conda-forge libcurand 10.3.1.50 hd3aeb46_1 conda-forge libcurand-dev 10.3.1.50 hd3aeb46_1 conda-forge libcusolver 11.4.2.57 hd3aeb46_2 conda-forge libcusolver-dev 11.4.2.57 hd3aeb46_2 conda-forge libcusparse 12.0.0.76 hd3aeb46_2 conda-forge libcusparse-dev 12.0.0.76 hd3aeb46_2 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-devel_linux-64 12.3.0 h8bca6fd_105 conda-forge libgcc-ng 13.2.0 h807b86a_5 conda-forge libgomp 13.2.0 h807b86a_5 conda-forge libnpp 12.0.0.30 hd3aeb46_1 conda-forge libnpp-dev 12.0.0.30 hd3aeb46_1 conda-forge libnsl 2.0.1 hd590300_0 conda-forge libnuma 2.0.18 hd590300_0 conda-forge libnvjitlink 12.0.76 hd3aeb46_2 conda-forge libnvjitlink-dev 12.0.76 hd3aeb46_2 conda-forge libnvjpeg 12.0.0.28 h59595ed_1 conda-forge libnvjpeg-dev 12.0.0.28 ha770c72_1 conda-forge libsanitizer 12.3.0 h0f45ef3_5 conda-forge libsodium 1.0.18 h36c2ea0_1 conda-forge libsqlite 3.45.2 h2797004_0 conda-forge libstdcxx-devel_linux-64 12.3.0 h8bca6fd_105 conda-forge libstdcxx-ng 13.2.0 h7e041cc_5 conda-forge libuuid 2.38.1 h0b41bf4_0 conda-forge libxcrypt 4.4.36 hd590300_1 conda-forge libzlib 1.2.13 hd590300_5 conda-forge lightning-utilities 0.3.0 pypi_0 pypi lmdb 1.4.1 pypi_0 pypi markdown 3.6 pypi_0 pypi markupsafe 2.1.5 pypi_0 pypi matplotlib 3.8.4 pypi_0 pypi matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge mpmath 1.3.0 pypi_0 pypi multidict 6.0.5 pypi_0 pypi ncurses 6.4.20240210 h59595ed_0 conda-forge nest-asyncio 1.6.0 pyhd8ed1ab_0 conda-forge networkx 3.2.1 pypi_0 pypi nsight-compute 2022.4.0.15 0 nvidia numpy 1.26.4 pypi_0 pypi nvidia-cublas-cu12 12.1.3.1 pypi_0 pypi nvidia-cuda-cupti-cu12 12.1.105 pypi_0 pypi nvidia-cuda-nvrtc-cu12 12.1.105 pypi_0 pypi nvidia-cuda-runtime-cu12 12.1.105 pypi_0 pypi nvidia-cudnn-cu12 8.9.2.26 pypi_0 pypi nvidia-cufft-cu12 11.0.2.54 pypi_0 pypi nvidia-curand-cu12 10.3.2.106 pypi_0 pypi nvidia-cusolver-cu12 11.4.5.107 pypi_0 pypi nvidia-cusparse-cu12 12.1.0.106 pypi_0 pypi nvidia-ml-py 11.495.46 pypi_0 pypi nvidia-nccl-cu12 2.19.3 pypi_0 pypi nvidia-nvjitlink-cu12 12.4.127 pypi_0 pypi nvidia-nvtx-cu12 12.1.105 pypi_0 pypi ocl-icd 2.3.2 hd590300_1 conda-forge omegaconf 2.3.0 pypi_0 pypi opencv-python 4.9.0.80 pypi_0 pypi openssl 3.2.1 hd590300_1 conda-forge packaging 24.0 pyhd8ed1ab_0 conda-forge pandas 2.2.1 pypi_0 pypi parso 0.8.3 pyhd8ed1ab_0 conda-forge pexpect 4.9.0 pyhd8ed1ab_0 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 10.3.0 pypi_0 pypi pip 24.0 pyhd8ed1ab_0 conda-forge platformdirs 4.2.0 pyhd8ed1ab_0 conda-forge pluggy 1.4.0 pypi_0 pypi pointnet2 0.0.0 pypi_0 pypi portalocker 2.8.2 pypi_0 pypi progressbar 2.5 pypi_0 pypi prompt-toolkit 3.0.42 pyha770c72_0 conda-forge protobuf 5.26.1 pypi_0 pypi psutil 5.9.8 py39hd1e30aa_0 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge pycocotools 2.0.7 pypi_0 pypi pyglet 2.0.15 pypi_0 pypi pygments 2.17.2 pyhd8ed1ab_0 conda-forge pynvml 11.5.0 pypi_0 pypi pyopengl 3.1.0 pypi_0 pypi pyparsing 3.1.2 pypi_0 pypi pyrender 0.1.45 pypi_0 pypi pysocks 1.7.1 pypi_0 pypi pytest 8.1.1 pypi_0 pypi python 3.9.19 h0755675_0_cpython conda-forge python-dateutil 2.9.0.post0 pypi_0 pypi python_abi 3.9 4_cp39 conda-forge pytorch-lightning 1.8.1 pypi_0 pypi pytz 2024.1 pypi_0 pypi pyyaml 6.0.1 pypi_0 pypi pyzmq 25.1.2 py39h8c080ef_0 conda-forge readline 8.2 h8228510_1 conda-forge requests 2.31.0 pypi_0 pypi ruamel-yaml 0.18.6 pypi_0 pypi ruamel-yaml-clib 0.2.8 pypi_0 pypi safetensors 0.4.2 pypi_0 pypi scikit-image 0.22.0 pypi_0 pypi scipy 1.13.0 pypi_0 pypi seaborn 0.13.2 pypi_0 pypi segment-anything 1.0 pypi_0 pypi setuptools 69.2.0 pyhd8ed1ab_0 conda-forge shapely 2.0.3 pypi_0 pypi six 1.16.0 pyh6c4a22f_0 conda-forge soupsieve 2.5 pypi_0 pypi sqlite 3.45.2 h2c6b66d_0 conda-forge stack_data 0.6.2 pyhd8ed1ab_0 conda-forge sympy 1.12 pypi_0 pypi sysroot_linux-64 2.17 h4a8ded7_14 conda-forge tabulate 0.9.0 pypi_0 pypi tensorboard 2.16.2 pypi_0 pypi tensorboard-data-server 0.7.2 pypi_0 pypi tensorboardx 2.6.2.2 pypi_0 pypi termcolor 2.4.0 pypi_0 pypi tifffile 2024.2.12 pypi_0 pypi timm 0.9.16 pypi_0 pypi tk 8.6.13 noxft_h4845f30_101 conda-forge tomli 2.0.1 pypi_0 pypi torch 2.2.1 pypi_0 pypi torchmetrics 0.11.4 pypi_0 pypi torchvision 0.17.1 pypi_0 pypi tornado 6.4 py39hd1e30aa_0 conda-forge tqdm 4.66.2 pypi_0 pypi traitlets 5.14.2 pyhd8ed1ab_0 conda-forge trimesh 4.0.8 pypi_0 pypi triton 2.2.0 pypi_0 pypi typing_extensions 4.10.0 pyha770c72_0 conda-forge tzdata 2024.1 pypi_0 pypi ultralytics 8.0.135 pypi_0 pypi urllib3 2.2.1 pypi_0 pypi wcwidth 0.2.13 pyhd8ed1ab_0 conda-forge werkzeug 3.0.2 pypi_0 pypi wheel 0.43.0 pyhd8ed1ab_1 conda-forge xformers 0.0.25 pypi_0 pypi xz 5.2.6 h166bdaf_0 conda-forge yacs 0.1.8 pypi_0 pypi yapf 0.40.2 pypi_0 pypi yarl 1.9.4 pypi_0 pypi zeromq 4.3.5 h59595ed_1 conda-forge zipp 3.18.1 pypi_0 pypi zlib 1.2.13 hd590300_5 conda-forge ```

By the way, when I try to run it on depth+rgb coming from an Intel Realsense, the algorithm doesn't seem to work properly. The segmentation works fine, but the pose estimation is completely messed up. On the other hand, if I run the algorithm on the same object but on a Blender-generated rgb+depth I get very good results.

Have you tried to run the algorithm on real data? My guess is that the algorithm does not work when there is no perfect correspondece between depth and rgb data (like in the Realsense)

traversaro commented 1 month ago

@matteomastrogiuseppe for future reproducibility, it would be great if you could report the mamba list of the environment in which everything is working for you. To avoid clutter the issue, you can use the <details> html tag, see:

<details>
<summary> click to see everything </summary>

```
this is hidden
```
</details>

that is rendered as:

you only see these ``` this is hidden ```
savidini commented 1 month ago

Thanks for your feedback @matteomastrogiuseppe! I am glad it worked for you too and I will create a PR later on.

Regarding the inference on real data: I just tested it, and for me it was the same at first. ISM worked, but PEM did not. Turns out it had something to do with my depth map. As you already mentioned, the correspondence between depth and RGB data must be exact. For this the frames need to be aligned.

Here is a somewhat detailed explanation of how it worked for me:

1. Get .ply for model, make sure units and scale are correct

Unfortunately, I don't have many objects available for which I also have CAD files, and vice versa. So I used a Rubik's Cube and a model I found online. I had to convert and scale it (I did this with Meshlab).

If you have a Rubik's Cube at hand, here is my .ply file: rubiks_cube.ply

2. Capture aligned RGB, depth frames and intrinsic data from Realsense

I am using a slightly modified script, see below, from librealsense to capture the aligned RGB and depth image, as well as my camera.json file with the intrinsic data from the Realsense D435 I am using. (Disclaimer: I am not a 100% sure if the intrinsic data is correct, as I am still quite new to the whole topic)

realsense_capture.py ```python ## License: Apache 2.0. See LICENSE file in root directory. ## Copyright(c) 2017 Intel Corporation. All Rights Reserved. ##################################################### ## Align Depth to Color ## ##################################################### # First import the library import pyrealsense2 as rs # Import Numpy for easy array manipulation import numpy as np # Import OpenCV for easy image rendering import cv2 # Import json to save camera instrinsics import json # Create a pipeline pipeline = rs.pipeline() # Create a config and configure the pipeline to stream # different resolutions of color and depth streams config = rs.config() # Get device product line for setting a supporting resolution pipeline_wrapper = rs.pipeline_wrapper(pipeline) pipeline_profile = config.resolve(pipeline_wrapper) device = pipeline_profile.get_device() device_product_line = str(device.get_info(rs.camera_info.product_line)) found_rgb = False for s in device.sensors: if s.get_info(rs.camera_info.name) == "RGB Camera": found_rgb = True break if not found_rgb: print("The demo requires Depth camera with Color sensor") exit(0) config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30) if device_product_line == "L500": config.enable_stream(rs.stream.color, 960, 540, rs.format.bgr8, 30) else: config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30) # Start streaming profile = pipeline.start(config) # Getting the depth sensor's depth scale (see rs-align example for explanation) depth_sensor = profile.get_device().first_depth_sensor() depth_scale = depth_sensor.get_depth_scale() print("Depth Scale is: ", depth_scale) # We will be removing the background of objects more than # clipping_distance_in_meters meters away clipping_distance_in_meters = 1 # 1 meter clipping_distance = clipping_distance_in_meters / depth_scale # Create an align object # rs.align allows us to perform alignment of depth frames to others frames # The "align_to" is the stream type to which we plan to align depth frames. align_to = rs.stream.color align = rs.align(align_to) # Streaming loop try: while True: # Get frameset of color and depth frames = pipeline.wait_for_frames() # frames.get_depth_frame() is a 640x360 depth image # Align the depth frame to color frame aligned_frames = align.process(frames) # Get aligned frames aligned_depth_frame = ( aligned_frames.get_depth_frame() ) # aligned_depth_frame is a 640x480 depth image color_frame = aligned_frames.get_color_frame() # Get instrinsics from aligned_depth_frame intrinsics = aligned_depth_frame.profile.as_video_stream_profile().intrinsics # print(intrinsics) # Validate that both frames are valid if not aligned_depth_frame or not color_frame: continue depth_image = np.asanyarray(aligned_depth_frame.get_data()) color_image = np.asanyarray(color_frame.get_data()) # Remove background - Set pixels further than clipping_distance to grey grey_color = 153 depth_image_3d = np.dstack( (depth_image, depth_image, depth_image) ) # depth image is 1 channel, color is 3 channels bg_removed = np.where( (depth_image_3d > clipping_distance) | (depth_image_3d <= 0), grey_color, color_image, ) # Render images: # depth align to color on left # depth on right depth_colormap = cv2.applyColorMap( cv2.convertScaleAbs(depth_image, alpha=0.03), cv2.COLORMAP_JET ) images = np.hstack((color_image, depth_colormap)) cv2.namedWindow("Align Example", cv2.WINDOW_NORMAL) cv2.imshow("Align Example", images) key = cv2.waitKey(1) # Press esc or 'q' to close the image window if key & 0xFF == ord("q") or key == 27: cv2.destroyAllWindows() cv2.imwrite("depth_test.png", depth_image) cv2.imwrite("rgb_test.png", color_image) # Get depth profile and camera intrinsics intrinsics_matrix = { "cam_K": [ intrinsics.fx, 0.0, intrinsics.ppx, 0.0, intrinsics.fy, intrinsics.ppy, 0.0, 0.0, 1.0, ], "depth_scale": depth_scale * 1000, } with open("camera.json", "w") as f: json.dump(intrinsics_matrix, f) break finally: pipeline.stop() ```

The script saves the color image and the depth map for the last frame when it is closed (Q/ESC).

My sample output data from the script:

rgb_test.png ![rgb_test](https://github.com/JiehongLin/SAM-6D/assets/42057206/57eadcea-5338-4d1f-afd3-fc8aabe7184b)
depth_test.png ![depth_test](https://github.com/JiehongLin/SAM-6D/assets/42057206/ef9004ca-4190-4221-b68a-27092cce9409)
camera.json ```json {"cam_K": [614.7604370117188, 0.0, 322.7101135253906, 0.0, 614.9583129882812, 233.1604766845703, 0.0, 0.0, 1.0], "depth_scale": 1.0000000474974513} ```

3. Set paths for custom data

Same as for the example data, run from directory containing all the files:

export CAD_PATH=$PWD/rubiks_cube_scaled.ply
export RGB_PATH=$PWD/rgb_test.png
export DEPTH_PATH=$PWD/depth_test.png
export CAMERA_PATH=$PWD/camera.json
export OUTPUT_DIR=$PWD/outputs

4. Run inference via sh demo.sh

Note that for me this often results in OOM errors (I am using one RTX4090/24GB VRAM). Possible solution, if only a small amount of memory is missing:

export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

Otherwise reduce points_per_side and points_per_batch in automatic_mask_generator.py (see #24). I am currently using the following:

        points_per_side: Optional[int] = 16,
        points_per_batch: int = 32,

(The required memory seems to be somewhat dependent on the size of the object in the image, not sure, very strange behavior.)

5. Results

vis_ism.png: vis_ism vis_pem.png: vis_pem

Hope this helps. Regards and have a nice weekend!

matteomastrogiuseppe commented 1 month ago

@savidini Infinite thanks for solving also this issue, aligning depth and RGB was indeed the key! Kind regards!

I'll keep this issue opened until the PR is accepted.

monajalal commented 1 month ago

@savidini I tried your environment.yaml and still got same error.

Could you please have a look?

I think prepare.sh is wrong?

dm-4.66.2 trimesh-4.0.8 triton-2.2.0 typing-extensions-4.11.0 tzdata-2024.1 ultralytics-8.0.135 urllib3-2.2.1 wcwidth-0.2.13 werkzeug-3.0.2 xformers-0.0.25 yacs-0.1.8 yapf-0.40.2 yarl-1.9.4 zipp-3.18.1

done
#
# To activate this environment, use
#
#     $ conda activate sam6d
#
# To deactivate an active environment, use
#
#     $ conda deactivate

CondaError: Run 'conda init' before 'conda activate'

Traceback (most recent call last):
  File "/data/SAM-6D/SAM-6D/Pose_Estimation_Model/model/pointnet2/setup.py", line 7, in <module>
    from torch.utils.cpp_extension import BuildExtension, CUDAExtension
ModuleNotFoundError: No module named 'torch'
Traceback (most recent call last):
  File "/data/SAM-6D/SAM-6D/Instance_Segmentation_Model/download_sam.py", line 10, in <module>
    import hydra
ModuleNotFoundError: No module named 'hydra'
Traceback (most recent call last):
  File "/data/SAM-6D/SAM-6D/Instance_Segmentation_Model/download_fastsam.py", line 10, in <module>
    import hydra
ModuleNotFoundError: No module named 'hydra'
Traceback (most recent call last):
  File "/data/SAM-6D/SAM-6D/Instance_Segmentation_Model/download_dinov2.py", line 10, in <module>
    import hydra
ModuleNotFoundError: No module named 'hydra'
sh: 1: gdown: not found
(base) mona@ada:/data/SAM-6D/SAM-6D$ sh prepare.sh
monajalal commented 1 month ago

also @savidini 

  File "/home/mona/anaconda3/envs/sam6d/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 523, in build_extensions
    _check_cuda_version(compiler_name, compiler_version)
  File "/home/mona/anaconda3/envs/sam6d/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 414, in _check_cuda_version
    raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
RuntimeError: 
The detected CUDA version (11.8) mismatches the version that was used to compile
PyTorch (12.1). Please make sure to use the same CUDA versions.
savidini commented 1 month ago

Hi @monajalal,

regarding the first issue, it looks like you are running the prepare.sh from the base conda environment.

Please note that my fork (#41) moves the creation and activation of the environment from the prepare.sh to a manual step (because this is more robust on remote systems). Please read the readme of my fork and follow the steps listed there, this should hopefully fix the problem. :smiley:

Regarding the CUDA mismatch, could you please copy the output of the nvidia-smi command on your machine? If you are using older NVIDIA drivers, it is possible that my fork will not work.

david1309 commented 1 month ago

Hi @savidini , thanks a lot for the improvements from your fork, it seems it's been really useful.

I tried using your fork, however, I quickly ran into issues just for creating the conda environment using the updated environment.yml file. When using the original file from the original repo, I was able to create the environment without issues, but then I ran into problems for installing PointNet, and thus I wanted to use your fork.

The issue I'm getting with your updated version is that creating the conda env takes ages, and then it crashes because it can't solve certain dependencies:

Package pip conflicts for:
pip
python=3.9.6 -> pipThe following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.31=0
  - feature:/linux-64::__linux==5.15.0=0
  - feature:|@/linux-64::__glibc==2.31=0
  - cuda=12.0 -> __linux
  - cuda=12.0 -> __win
  - python=3.9.6 -> libgcc-ng[version='>=9.3.0'] -> __glibc[version='>=2.17']

Your installed version is: 2.31

Here are some details of my server's setup:

conda information:

conda version : 23.3.1
conda-build version : 3.23.3
python version : 3.9.19.final.0
virtual packages : __archspec=1=x86_64
                          __cuda=12.2=0
                          __glibc=2.31=0
                          __linux=5.15.0=0
                          __unix=0=0

nvidia-smi output:

 NVIDIA-SMI 535.171.04             Driver Version: 535.171.04   CUDA Version: 12.2

nvcc Driver output:

Cuda compilation tools, release 11.1, V11.1.74
Build cuda_11.1.TC455_06.29069683_

I'm not sure what's the issue, but it might be because I have __glibc=2.31=0 (I'm not sure what this does). I commented out the cuda=12.0 and libxcrypt from the .yaml file, and with this, I solved the environment creation issue. However, when I run the prepare.sh and try to compile PointNet, it fails because of this: 

RuntimeError: 
The detected CUDA version (11.1) mismatches the version that was used to compile
PyTorch (12.1). Please make sure to use the same CUDA versions.
savidini commented 1 month ago

Hi @david1309

The issue I'm getting with your updated version is that creating the conda env takes ages, and then it crashes because it can't solve certain dependencies

Take forever and fail to solve the environment: This sounds a lot like the old conda solver that your conda version still uses by default. :sweat_smile:

Can you try creating the environment with the libamamba solver instead? Either by following these instructions, or by installing a newer version of conda.

It should take about 5 minutes to solve and install the environment. :wink:

Let me know if it works.

david1309 commented 1 month ago

Hi @savidini , thanks for your prompt response.

Indeed, part of the issue might come from the old conda, so as you suggested I updated conda and also set the default solver to be libmamba. I also manually updated from NVIDIA’s CUDA website the CUDA to 12.1 (since I also had the old 11.1 one)

Having updated conda and CUDA, I re-tried creating your fork's environment, but I got the following error when trying to install cuda. At first I tried with cuda=12.0 but it produced the error, so I changed it to cuda=12.1 since that's the version of I manually installed, but it still produced the same error:

CUDA environment creation error Screenshot 2024-04-18 at 11 37 27

I then created the environment but commented out the cuda package, which successfully created the environment. However, when trying to compile PointNet, there's an error as the compiler can't manage to find the header files:

PointNet compilation error ```bash self.initialize_options() running bdist_egg running egg_info writing pointnet2.egg-info/PKG-INFO writing dependency_links to pointnet2.egg-info/dependency_links.txt writing top-level names to pointnet2.egg-info/top_level.txt reading manifest file 'pointnet2.egg-info/SOURCES.txt' writing manifest file 'pointnet2.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_ext /hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/lib/python3.9/site-packages/torch/utils/cpp_extension.py:425: UserWarning: There are no g++ version bounds defined for CUDA version 12.1 warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}') building 'pointnet2._ext' extension Emitting ninja build file /home/dalvarez/Documents/sam_6d/sam_6d/models/Pose_Estimation_Model/model/pointnet2/build/temp.linux-x86_64-cpython-39/build.ninja... Compiling objects... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [1/9] c++ -MMD -MF /home/dalvarez/Documents/sam_6d/sam_6d/models/Pose_Estimation_Model/model/pointnet2/build/temp.linux-x86_64-cpython-39/_ext_src/src/ball_query.o.d -pthread -B /hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/include -fPIC -O2 -isystem /hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/include -fPIC -I_ext_src/include -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/lib/python3.9/site-packages/torch/include -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/lib/python3.9/site-packages/torch/include/TH -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/lib/python3.9/site-packages/torch/include/THC -I/home/dalvarez/cuda/cuda-12.1/include -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/include/python3.9 -c -c /home/dalvarez/Documents/sam_6d/sam_6d/models/Pose_Estimation_Model/model/pointnet2/_ext_src/src/ball_query.cpp -o /home/dalvarez/Documents/sam_6d/sam_6d/models/Pose_Estimation_Model/model/pointnet2/build/temp.linux-x86_64-cpython-39/_ext_src/src/ball_query.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17 FAILED: /home/dalvarez/Documents/sam_6d/sam_6d/models/Pose_Estimation_Model/model/pointnet2/build/temp.linux-x86_64-cpython-39/_ext_src/src/ball_query.o c++ -MMD -MF /home/dalvarez/Documents/sam_6d/sam_6d/models/Pose_Estimation_Model/model/pointnet2/build/temp.linux-x86_64-cpython-39/_ext_src/src/ball_query.o.d -pthread -B /hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/include -fPIC -O2 -isystem /hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/include -fPIC -I_ext_src/include -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/lib/python3.9/site-packages/torch/include -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/lib/python3.9/site-packages/torch/include/TH -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/lib/python3.9/site-packages/torch/include/THC -I/home/dalvarez/cuda/cuda-12.1/include -I/hdd1/opt/anaconda3/envs/sam6d_savidini_fork_2/include/python3.9 -c -c /home/dalvarez/Documents/sam_6d/sam_6d/models/Pose_Estimation_Model/model/pointnet2/_ext_src/src/ball_query.cpp -o /home/dalvarez/Documents/sam_6d/sam_6d/models/Pose_Estimation_Model/model/pointnet2/build/temp.linux-x86_64-cpython-39/_ext_src/src/ball_query.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17 /home/dalvarez/Documents/sam_6d/sam_6d/models/Pose_Estimation_Model/model/pointnet2/_ext_src/src/ball_query.cpp:6**:10: fatal error: ball_query.h: No such file or directory 6 | #include "ball_query.h"** ```

In particular I see these two error: .../python3.9/site-packages/torch/utils/cpp_extension.py:425: UserWarning: There are no g++ version bounds defined for CUDA version 12.1 and 10: fatal error: ball_query.h: No such file or directory

Any guidance ?

savidini commented 1 month ago

Hi @david1309,

I don't think it should be necessary to install CUDA manually. Your NVIDIA driver supports CUDA up to version 12.2, which is fine. CUDA and its components can be installed directly into conda, which in my experience is also the most robust when it comes to solving the environment.

Your error in creating the CUDA environment seems very strange. It seems that conda can only find a Windows version. Are you running WSL Ubuntu by any chance? I have never seen such an error in conda before. :raised_eyebrow:

Can you please provide the output of the following commands (from anywhere/base env): 

cat /etc/os-release
conda info
echo $CUDA_PATH
echo $CUDA_HOME
echo $PYTHONPATH

And from your activated sam6d environment (e.g. sam6d_savidini_fork_2):

conda list
which nvcc
which python
which pip

Maybe I can try to find a solution during this weekend. :v:

david1309 commented 1 month ago

Hi @savidini , thanks for your kind help.

Here are the outputs from the base environment: **cat /etc/os-release** NAME="Ubuntu" VERSION="20.04.6 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.6 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal **conda info** active environment : base active env location : /hdd1/opt/anaconda3 shell level : 3 user config file : /home/dalvarez/.condarc populated config files : /home/dalvarez/.condarc conda version : 24.3.0 conda-build version : 24.3.0 python version : 3.9.19.final.0 solver : libmamba (default) virtual packages : __archspec=1=zen2 __conda=24.3.0=0 __cuda=12.2=0 __glibc=2.31=0 __linux=5.15.0=0 __unix=0=0 base environment : /hdd1/opt/anaconda3 (writable) conda av data dir : /hdd1/opt/anaconda3/etc/conda conda av metadata url : None channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /hdd1/opt/anaconda3/pkgs /home/dalvarez/.conda/pkgs envs directories : /hdd1/opt/anaconda3/envs /home/dalvarez/.conda/envs platform : linux-64 user-agent : conda/24.3.0 requests/2.31.0 CPython/3.9.19 Linux/5.15.0-102-generic ubuntu/20.04.6 glibc/2.31 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8 UID:GID : 1005:1006 netrc file : /home/dalvarez/.netrc offline mode : False **echo $CUDA_PATH** **echo $CUDA_HOME** /home/dalvarez/cuda/cuda-12.1/ **echo $PYTHONPATH** **echo $PATH** /home/dalvarez/cuda/cuda-12.1/bin:/hdd1/opt/anaconda3/bin:/hdd1/opt/anaconda3/condabin

The environment called sam6d_savidini_fork was created using your fork's environment.yaml but commenting out the cuda=12.0 package, given that if I included that package the environment would fail to install.

Here are the outputs from the sam6d_savidini_fork environment : **conda list** ```bash # packages in environment at /hdd1/opt/anaconda3/envs/sam6d_savidini_fork: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge absl-py 2.1.0 pypi_0 pypi addict 2.4.0 pypi_0 pypi aiohttp 3.9.5 pypi_0 pypi aiosignal 1.3.1 pypi_0 pypi antlr4-python3-runtime 4.9.3 pypi_0 pypi async-timeout 4.0.3 pypi_0 pypi attrs 23.2.0 pypi_0 pypi beautifulsoup4 4.12.3 pypi_0 pypi blenderproc 2.7.0 pypi_0 pypi blessed 1.20.0 pypi_0 pypi ca-certificates 2024.2.2 hbcca054_0 conda-forge certifi 2024.2.2 pypi_0 pypi charset-normalizer 3.3.2 pypi_0 pypi colorlog 6.8.2 pypi_0 pypi contourpy 1.2.1 pypi_0 pypi cycler 0.12.1 pypi_0 pypi distinctipy 1.3.4 pypi_0 pypi einops 0.7.0 pypi_0 pypi exceptiongroup 1.2.1 pypi_0 pypi filelock 3.13.4 pypi_0 pypi fire 0.6.0 pypi_0 pypi fonttools 4.51.0 pypi_0 pypi freetype-py 2.4.0 pypi_0 pypi frozenlist 1.4.1 pypi_0 pypi fsspec 2024.3.1 pypi_0 pypi fvcore 0.1.5.post20221221 pypi_0 pypi gdown 5.1.0 pypi_0 pypi gorilla-core 0.2.7.8 pypi_0 pypi gpustat 1.0.0 pypi_0 pypi grpcio 1.62.2 pypi_0 pypi h5py 3.11.0 pypi_0 pypi huggingface-hub 0.22.2 pypi_0 pypi hydra-colorlog 1.2.0 pypi_0 pypi hydra-core 1.3.2 pypi_0 pypi idna 3.7 pypi_0 pypi imageio 2.34.0 pypi_0 pypi imgaug 0.4.0 pypi_0 pypi importlib-metadata 7.1.0 pypi_0 pypi importlib-resources 6.4.0 pypi_0 pypi iniconfig 2.0.0 pypi_0 pypi iopath 0.1.10 pypi_0 pypi jinja2 3.1.3 pypi_0 pypi kiwisolver 1.4.5 pypi_0 pypi lazy-loader 0.4 pypi_0 pypi ld_impl_linux-64 2.40 h41732ed_0 conda-forge libffi 3.3 h58526e2_2 conda-forge libgcc-ng 13.2.0 h807b86a_5 conda-forge libgomp 13.2.0 h807b86a_5 conda-forge libsqlite 3.45.3 h2797004_0 conda-forge libstdcxx-ng 13.2.0 h7e041cc_5 conda-forge libxcrypt 4.4.36 hd590300_1 conda-forge libzlib 1.2.13 hd590300_5 conda-forge lightning-utilities 0.3.0 pypi_0 pypi lmdb 1.4.1 pypi_0 pypi markdown 3.6 pypi_0 pypi markupsafe 2.1.5 pypi_0 pypi matplotlib 3.8.4 pypi_0 pypi mpmath 1.3.0 pypi_0 pypi multidict 6.0.5 pypi_0 pypi ncurses 6.4.20240210 h59595ed_0 conda-forge networkx 3.2.1 pypi_0 pypi numpy 1.26.4 pypi_0 pypi nvidia-cublas-cu12 12.1.3.1 pypi_0 pypi nvidia-cuda-cupti-cu12 12.1.105 pypi_0 pypi nvidia-cuda-nvrtc-cu12 12.1.105 pypi_0 pypi nvidia-cuda-runtime-cu12 12.1.105 pypi_0 pypi nvidia-cudnn-cu12 8.9.2.26 pypi_0 pypi nvidia-cufft-cu12 11.0.2.54 pypi_0 pypi nvidia-curand-cu12 10.3.2.106 pypi_0 pypi nvidia-cusolver-cu12 11.4.5.107 pypi_0 pypi nvidia-cusparse-cu12 12.1.0.106 pypi_0 pypi nvidia-ml-py 11.495.46 pypi_0 pypi nvidia-nccl-cu12 2.19.3 pypi_0 pypi nvidia-nvjitlink-cu12 12.4.127 pypi_0 pypi nvidia-nvtx-cu12 12.1.105 pypi_0 pypi omegaconf 2.3.0 pypi_0 pypi opencv-python 4.9.0.80 pypi_0 pypi openssl 1.1.1w hd590300_0 conda-forge packaging 24.0 pypi_0 pypi pandas 2.2.2 pypi_0 pypi pillow 10.3.0 pypi_0 pypi pip 24.0 pyhd8ed1ab_0 conda-forge platformdirs 4.2.0 pypi_0 pypi pluggy 1.4.0 pypi_0 pypi portalocker 2.8.2 pypi_0 pypi progressbar 2.5 pypi_0 pypi protobuf 5.26.1 pypi_0 pypi psutil 5.9.8 pypi_0 pypi pycocotools 2.0.7 pypi_0 pypi pyglet 2.0.15 pypi_0 pypi pynvml 11.5.0 pypi_0 pypi pyopengl 3.1.0 pypi_0 pypi pyparsing 3.1.2 pypi_0 pypi pyrender 0.1.45 pypi_0 pypi pysocks 1.7.1 pypi_0 pypi pytest 8.1.1 pypi_0 pypi python 3.9.6 h49503c6_1_cpython conda-forge python-dateutil 2.9.0.post0 pypi_0 pypi pytorch-lightning 1.8.1 pypi_0 pypi pytz 2024.1 pypi_0 pypi pyyaml 6.0.1 pypi_0 pypi readline 8.2 h8228510_1 conda-forge requests 2.31.0 pypi_0 pypi ruamel-yaml 0.18.6 pypi_0 pypi ruamel-yaml-clib 0.2.8 pypi_0 pypi safetensors 0.4.3 pypi_0 pypi scikit-image 0.22.0 pypi_0 pypi scipy 1.13.0 pypi_0 pypi seaborn 0.13.2 pypi_0 pypi segment-anything 1.0 pypi_0 pypi setuptools 69.5.1 pyhd8ed1ab_0 conda-forge shapely 2.0.4 pypi_0 pypi six 1.16.0 pypi_0 pypi soupsieve 2.5 pypi_0 pypi sqlite 3.45.3 h2c6b66d_0 conda-forge sympy 1.12 pypi_0 pypi tabulate 0.9.0 pypi_0 pypi tensorboard 2.16.2 pypi_0 pypi tensorboard-data-server 0.7.2 pypi_0 pypi tensorboardx 2.6.2.2 pypi_0 pypi termcolor 2.4.0 pypi_0 pypi tifffile 2024.4.18 pypi_0 pypi timm 0.9.16 pypi_0 pypi tk 8.6.13 noxft_h4845f30_101 conda-forge tomli 2.0.1 pypi_0 pypi torch 2.2.1 pypi_0 pypi torchmetrics 0.11.4 pypi_0 pypi torchvision 0.17.1 pypi_0 pypi tqdm 4.66.2 pypi_0 pypi trimesh 4.0.8 pypi_0 pypi triton 2.2.0 pypi_0 pypi typing-extensions 4.11.0 pypi_0 pypi tzdata 2024.1 pypi_0 pypi ultralytics 8.0.135 pypi_0 pypi urllib3 2.2.1 pypi_0 pypi wcwidth 0.2.13 pypi_0 pypi werkzeug 3.0.2 pypi_0 pypi wheel 0.43.0 pyhd8ed1ab_1 conda-forge xformers 0.0.25 pypi_0 pypi xz 5.2.6 h166bdaf_0 conda-forge yacs 0.1.8 pypi_0 pypi yapf 0.40.2 pypi_0 pypi yarl 1.9.4 pypi_0 pypi zipp 3.18.1 pypi_0 pypi zlib 1.2.13 hd590300_5 conda-forge ``` **which nvcc** /home/dalvarez/cuda/cuda-12.1/bin/nvcc **which python** /hdd1/opt/anaconda3/envs/sam6d_savidini_fork/bin/python **which pip** /hdd1/opt/anaconda3/envs/sam6d_savidini_fork/bin/pip
savidini commented 1 month ago

Hi @david1309, long story short: I think in your case the local CUDA installation leads to conflicts, and my original environment.yaml was a bit buggy because PyTorch also tries to install it's own version of CUDA, see this nice post. This causes errors when building PointNet2.

You can try to create a new environment from the following file (this will not install CUDA, as you already have 12.1 installed):

environment.yaml ```yaml name: sam6d channels: - defaults - pytorch - nvidia - conda-forge dependencies: - pip - python=3.9 - pytorch::pytorch-cuda=12.1 - pytorch::pytorch - pytorch::torchvision - pytorch::torchaudio - conda-forge::torchmetrics - pip: - fvcore - xformers - blenderproc - opencv-python # ISM - omegaconf - ruamel.yaml - hydra-colorlog - hydra-core - gdown - pandas - imageio - pyrender - pytorch-lightning - pycocotools - distinctipy - git+https://github.com/facebookresearch/segment-anything.git # SAM - ultralytics # FastSAM # PEM - timm - gorilla-core - trimesh - gpustat - imgaug - einops ```

(You may want to run conda clean --all -y before creating the environment)

Let me know how it works. :v:

david1309 commented 1 month ago

I managed to fix the PointNet compilation error in which the compiler didn't find the .h header files. The reason is (probably) because my machine has an older version of gcc version 9.4.0 and it's not able to find the header files located under _ext_src.

The fix consists in modifying the file Pose_Estimation_Model/model/pointnet2/**setup.py**. Instead of referencing the header files with _ext_src_root = "_ext_src", I replaced this line by:

from pathlib import Path

_ext_src_root = str(Path(__file__).parent / "_ext_src")

Using this absolute path instead of the relative one seems to avoid the issue.