If it’s not a bug, please use discussions: https://github.com/NVlabs/curobo/discussions

Please provide the below information in addition to your issue:

cuRobo installation mode (choose from [python, isaac sim, docker python, docker isaac sim]): python
python version: 3.8.10
Isaac Sim version (if using): N/A
Ubuntu version: 20.04
GPU: NVIDIA GeForce RTX 4090
Driver Version: 525.125.06
CUDA version: 11.7

Hello Balakumar @balakumar-s ,

Thanks for your contributions to this library. I am getting an error when trying to install cuRobo (Python version without Isaac Sim). Following is the installation details. Only a few example can be executed normally, and the pytest cannot pass normally.

My setup is

conda create -n curobo-env python=3.8.10
conda activate curobo-env
conda install pytorch==1.13.0 torchvision==0.14.0 torchaudio==0.13.0 pytorch-cuda=11.7 -c pytorch -c nvidia
conda install pip
mkdir -p /home/pren/Space/GitHub-Codes/curobo_ws/curobo
cd /home/pren/Space/GitHub-Codes/curobo_ws/curobo
git clone https://github.com/NVlabs/curobo.git
cd curobo
git lfs pull
pip install -e . --no-build-isolation
python examples/collision_check_example.py  # Passed
python examples/ik_example.py  # Error

pip install pluggy
python3 -m pytest .  # 2 skipped, 5 warnings, 2 errors

Issue Details

(base) ➜  ~ conda create -n curobo-env python=3.8.10
Collecting package metadata (current_repodata.json): done
Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 23.7.4
  latest version: 23.11.0

Please update conda by running

    $ conda update -n base -c defaults conda

Or to minimize the number of packages updated during conda update use

     conda install conda=23.11.0

## Package Plan ##

  environment location: /home/pren/anaconda3/envs/curobo-env

  added / updated specs:
    - python=3.8.10

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main 
  _openmp_mutex      pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu 
  ca-certificates    pkgs/main/linux-64::ca-certificates-2023.12.12-h06a4308_0 
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1 
  libffi             pkgs/main/linux-64::libffi-3.3-he6710b0_2 
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1 
  libgomp            pkgs/main/linux-64::libgomp-11.2.0-h1234567_1 
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1 
  ncurses            pkgs/main/linux-64::ncurses-6.4-h6a678d5_0 
  openssl            pkgs/main/linux-64::openssl-1.1.1w-h7f8727e_0 
  pip                pkgs/main/linux-64::pip-23.3.1-py38h06a4308_0 
  python             pkgs/main/linux-64::python-3.8.10-h12debd9_8 
  readline           pkgs/main/linux-64::readline-8.2-h5eee18b_0 
  setuptools         pkgs/main/linux-64::setuptools-68.2.2-py38h06a4308_0 
  sqlite             pkgs/main/linux-64::sqlite-3.41.2-h5eee18b_0 
  tk                 pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0 
  wheel              pkgs/main/linux-64::wheel-0.41.2-py38h06a4308_0 
  xz                 pkgs/main/linux-64::xz-5.4.5-h5eee18b_0 
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0 

Proceed ([y]/n)? Y

Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate curobo-env
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) ➜  ~ conda activate curobo-env
(curobo-env) ➜  ~ conda install pytorch==1.13.0 torchvision==0.14.0 torchaudio==0.13.0 pytorch-cuda=11.7 -c pytorch -c nvidia
Collecting package metadata (current_repodata.json): done
Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 23.7.4
  latest version: 23.11.0

Please update conda by running

    $ conda update -n base -c defaults conda

Or to minimize the number of packages updated during conda update use

     conda install conda=23.11.0

## Package Plan ##

  environment location: /home/pren/anaconda3/envs/curobo-env

  added / updated specs:
    - pytorch-cuda=11.7
    - pytorch==1.13.0
    - torchaudio==0.13.0
    - torchvision==0.14.0

The following NEW packages will be INSTALLED:

  blas               pkgs/main/linux-64::blas-1.0-mkl 
  brotli-python      pkgs/main/linux-64::brotli-python-1.0.9-py38h6a678d5_7 
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h7b6447c_0 
  certifi            pkgs/main/linux-64::certifi-2023.11.17-py38h06a4308_0 
  cffi               pkgs/main/linux-64::cffi-1.15.1-py38h74dc2b5_0 
  charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0 
  cryptography       pkgs/main/linux-64::cryptography-41.0.3-py38h130f0dd_0 
  cuda-cudart        nvidia/linux-64::cuda-cudart-11.7.99-0 
  cuda-cupti         nvidia/linux-64::cuda-cupti-11.7.101-0 
  cuda-libraries     nvidia/linux-64::cuda-libraries-11.7.1-0 
  cuda-nvrtc         nvidia/linux-64::cuda-nvrtc-11.7.99-0 
  cuda-nvtx          nvidia/linux-64::cuda-nvtx-11.7.91-0 
  cuda-runtime       nvidia/linux-64::cuda-runtime-11.7.1-0 
  ffmpeg             pytorch/linux-64::ffmpeg-4.3-hf484d3e_0 
  freetype           pkgs/main/linux-64::freetype-2.12.1-h4a9f257_0 
  giflib             pkgs/main/linux-64::giflib-5.2.1-h5eee18b_3 
  gmp                pkgs/main/linux-64::gmp-6.2.1-h295c915_3 
  gnutls             pkgs/main/linux-64::gnutls-3.6.15-he1e5248_0 
  idna               pkgs/main/linux-64::idna-3.4-py38h06a4308_0 
  intel-openmp       pkgs/main/linux-64::intel-openmp-2021.4.0-h06a4308_3561 
  jpeg               pkgs/main/linux-64::jpeg-9e-h5eee18b_1 
  lame               pkgs/main/linux-64::lame-3.100-h7b6447c_0 
  lcms2              pkgs/main/linux-64::lcms2-2.12-h3be6417_0 
  lerc               pkgs/main/linux-64::lerc-3.0-h295c915_0 
  libcublas          nvidia/linux-64::libcublas-11.10.3.66-0 
  libcufft           nvidia/linux-64::libcufft-10.7.2.124-h4fbf590_0 
  libcufile          nvidia/linux-64::libcufile-1.8.1.2-0 
  libcurand          nvidia/linux-64::libcurand-10.3.4.101-0 
  libcusolver        nvidia/linux-64::libcusolver-11.4.0.1-0 
  libcusparse        nvidia/linux-64::libcusparse-11.7.4.91-0 
  libdeflate         pkgs/main/linux-64::libdeflate-1.17-h5eee18b_1 
  libiconv           pkgs/main/linux-64::libiconv-1.16-h7f8727e_2 
  libidn2            pkgs/main/linux-64::libidn2-2.3.4-h5eee18b_0 
  libnpp             nvidia/linux-64::libnpp-11.7.4.75-0 
  libnvjpeg          nvidia/linux-64::libnvjpeg-11.8.0.2-0 
  libpng             pkgs/main/linux-64::libpng-1.6.39-h5eee18b_0 
  libtasn1           pkgs/main/linux-64::libtasn1-4.19.0-h5eee18b_0 
  libtiff            pkgs/main/linux-64::libtiff-4.5.1-h6a678d5_0 
  libunistring       pkgs/main/linux-64::libunistring-0.9.10-h27cfd23_0 
  libwebp            pkgs/main/linux-64::libwebp-1.3.2-h11a3e52_0 
  libwebp-base       pkgs/main/linux-64::libwebp-base-1.3.2-h5eee18b_0 
  lz4-c              pkgs/main/linux-64::lz4-c-1.9.4-h6a678d5_0 
  mkl                pkgs/main/linux-64::mkl-2021.4.0-h06a4308_640 
  mkl-service        pkgs/main/linux-64::mkl-service-2.4.0-py38h7f8727e_0 
  mkl_fft            pkgs/main/linux-64::mkl_fft-1.3.1-py38hd3c417c_0 
  mkl_random         pkgs/main/linux-64::mkl_random-1.2.2-py38h51133e4_0 
  nettle             pkgs/main/linux-64::nettle-3.7.3-hbbd107a_1 
  numpy              pkgs/main/linux-64::numpy-1.24.3-py38h14f4228_0 
  numpy-base         pkgs/main/linux-64::numpy-base-1.24.3-py38h31eccc5_0 
  openh264           pkgs/main/linux-64::openh264-2.1.1-h4ff587b_0 
  openjpeg           pkgs/main/linux-64::openjpeg-2.4.0-h3ad879b_0 
  pillow             pkgs/main/linux-64::pillow-10.0.1-py38ha6cbd5a_0 
  pycparser          pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0 
  pyopenssl          pkgs/main/linux-64::pyopenssl-23.2.0-py38h06a4308_0 
  pysocks            pkgs/main/linux-64::pysocks-1.7.1-py38h06a4308_0 
  pytorch            pytorch/linux-64::pytorch-1.13.0-py3.8_cuda11.7_cudnn8.5.0_0 
  pytorch-cuda       pytorch/linux-64::pytorch-cuda-11.7-h778d358_5 
  pytorch-mutex      pytorch/noarch::pytorch-mutex-1.0-cuda 
  requests           pkgs/main/linux-64::requests-2.31.0-py38h06a4308_0 
  six                pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1 
  torchaudio         pytorch/linux-64::torchaudio-0.13.0-py38_cu117 
  torchvision        pytorch/linux-64::torchvision-0.14.0-py38_cu117 
  typing_extensions  pkgs/main/linux-64::typing_extensions-4.7.1-py38h06a4308_0 
  urllib3            pkgs/main/linux-64::urllib3-1.26.18-py38h06a4308_0 
  zstd               pkgs/main/linux-64::zstd-1.5.5-hc292b87_0 

Proceed ([y]/n)? Y

Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(curobo-env) ➜  ~ conda install pip
Collecting package metadata (current_repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 23.7.4
  latest version: 23.11.0

Please update conda by running

    $ conda update -n base -c defaults conda

Or to minimize the number of packages updated during conda update use

     conda install conda=23.11.0

# All requested packages already installed.

(curobo-env) ➜  ~ mkdir -p /home/pren/Space/GitHub-Codes/curobo_ws/curobo
(curobo-env) ➜  ~ cd /home/pren/Space/GitHub-Codes/curobo_ws/curobo
(curobo-env) ➜  curobo git clone https://github.com/NVlabs/curobo.git
Cloning into 'curobo'...
remote: Enumerating objects: 854, done.
remote: Counting objects: 100% (74/74), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 854 (delta 52), reused 43 (delta 36), pack-reused 780
Receiving objects: 100% (854/854), 2.68 MiB | 156.00 KiB/s, done.
Resolving deltas: 100% (316/316), done.
Filtering content: 100% (214/214), 238.55 MiB | 1.11 MiB/s, done.
(curobo-env) ➜  curobo cd curobo
(curobo-env) ➜  curobo git:(main) git lfs pull
(curobo-env) ➜  curobo git:(main) pip install -e . --no-build-isolation
Looking in indexes: https://mirrors.bfsu.edu.cn/pypi/web/simple/
Obtaining file:///home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo
  Checking if build backend supports build_editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: networkx in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (3.1)
Requirement already satisfied: numpy in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (1.22.0)
Requirement already satisfied: numpy-quaternion in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (2022.4.3)
Requirement already satisfied: pyyaml in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (6.0.1)
Requirement already satisfied: setuptools-scm>=6.2 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (8.0.4)
Requirement already satisfied: torchtyping in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (0.1.4)
Requirement already satisfied: torch>=1.10 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (1.12.1+cu113)
Requirement already satisfied: trimesh in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (3.15.5)
Requirement already satisfied: yourdfpy>=0.0.53 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (0.0.53)
Requirement already satisfied: warp-lang>=0.9.0 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (0.10.1)
Requirement already satisfied: scipy>=1.7.0 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (1.10.1)
Requirement already satisfied: tqdm in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (4.66.1)
Requirement already satisfied: wheel in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (0.41.2)
Requirement already satisfied: importlib-resources in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (6.1.0)
Requirement already satisfied: packaging>=20 in /home/pren/.local/lib/python3.8/site-packages (from setuptools-scm>=6.2->nvidia-curobo==0.6.2.post1.dev3) (23.2)
Requirement already satisfied: setuptools in /home/pren/.local/lib/python3.8/site-packages (from setuptools-scm>=6.2->nvidia-curobo==0.6.2.post1.dev3) (65.7.0)
Requirement already satisfied: typing-extensions in /home/pren/.local/lib/python3.8/site-packages (from setuptools-scm>=6.2->nvidia-curobo==0.6.2.post1.dev3) (4.8.0)
Requirement already satisfied: tomli>=1 in /home/pren/.local/lib/python3.8/site-packages (from setuptools-scm>=6.2->nvidia-curobo==0.6.2.post1.dev3) (2.0.1)
Collecting lxml (from yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3)
  Using cached https://mirrors.bfsu.edu.cn/pypi/web/packages/c3/4c/556867cb2193b323728678f239ce24463ae71b8636aba7d6e55a72e3650e/lxml-4.9.4-cp38-cp38-manylinux_2_28_x86_64.whl (8.0 MB)
Requirement already satisfied: six in /home/pren/anaconda3/envs/curobo-env/lib/python3.8/site-packages (from yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.16.0)
Requirement already satisfied: zipp>=3.1.0 in /home/pren/.local/lib/python3.8/site-packages (from importlib-resources->nvidia-curobo==0.6.2.post1.dev3) (3.17.0)
Requirement already satisfied: typeguard>=2.11.1 in /home/pren/.local/lib/python3.8/site-packages (from torchtyping->nvidia-curobo==0.6.2.post1.dev3) (4.1.5)
Requirement already satisfied: pillow in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (10.1.0)
Requirement already satisfied: requests in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2.31.0)
Collecting chardet (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3)
  Using cached https://mirrors.bfsu.edu.cn/pypi/web/packages/38/6f/f5fbc992a329ee4e0f288c1fe0e2ad9485ed064cac731ed2fe47dcc38cbf/chardet-5.2.0-py3-none-any.whl (199 kB)
Requirement already satisfied: msgpack in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.0.7)
Requirement already satisfied: xxhash in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (3.4.1)
Requirement already satisfied: shapely in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2.0.2)
Requirement already satisfied: sympy in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.12)
Requirement already satisfied: jsonschema in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (4.19.1)
Requirement already satisfied: mapbox-earcut in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.0.1)
Requirement already satisfied: pyglet in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.5.27)
Requirement already satisfied: rtree in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.1.0)
Requirement already satisfied: pycollada in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (0.7.2)
Requirement already satisfied: colorlog in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (6.7.0)
Requirement already satisfied: svg.path in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (6.3)
Requirement already satisfied: importlib-metadata>=3.6 in /home/pren/.local/lib/python3.8/site-packages (from typeguard>=2.11.1->torchtyping->nvidia-curobo==0.6.2.post1.dev3) (6.8.0)
Requirement already satisfied: attrs>=22.2.0 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (23.1.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2023.7.1)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.3.10)
Requirement already satisfied: referencing>=0.28.4 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (0.30.2)
Requirement already satisfied: rpds-py>=0.7.1 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (0.10.6)
Requirement already satisfied: python-dateutil>=2.2 in /home/pren/.local/lib/python3.8/site-packages (from pycollada->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2.8.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/pren/.local/lib/python3.8/site-packages (from requests->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (3.3.0)
Requirement already satisfied: idna<4,>=2.5 in /home/pren/anaconda3/envs/curobo-env/lib/python3.8/site-packages (from requests->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/pren/.local/lib/python3.8/site-packages (from requests->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /home/pren/anaconda3/envs/curobo-env/lib/python3.8/site-packages (from requests->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2023.11.17)
Requirement already satisfied: mpmath>=0.19 in /home/pren/.local/lib/python3.8/site-packages (from sympy->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.3.0)
Building wheels for collected packages: nvidia-curobo
  Building editable for nvidia-curobo (pyproject.toml) ... done
  Created wheel for nvidia-curobo: filename=nvidia_curobo-0.6.2.post1.dev3-0.editable-cp38-cp38-linux_x86_64.whl size=5627 sha256=08dfec994a0ee252ed2896f380cb0afa115d1af0c42b13d253b09afbf99066fe
  Stored in directory: /tmp/pip-ephem-wheel-cache-3v4y00ct/wheels/ac/e5/36/52e2c21d4f654260867554ec1f86a8445280b02c7854b2a741
Successfully built nvidia-curobo
Installing collected packages: lxml, chardet, nvidia-curobo
Successfully installed chardet-5.2.0 lxml-4.9.4 nvidia-curobo-0.6.2.post1.dev3

[notice] A new release of pip is available: 23.3.1 -> 23.3.2
[notice] To update, run: pip install --upgrade pip
(curobo-env) ➜  curobo git:(main) python examples/collision_check_example.py
/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/types/math.py:83: UserWarning: FALLBACK path has been taken inside: runCudaFusionGroup. This is an indication that codegen Failed for some reason.
To debug try disable codegen fallback path via setting the env variable `export PYTORCH_NVFUSER_DISABLE=fallback`
 (Triggered internally at  ../torch/csrc/jit/codegen/cuda/manager.cpp:329.)
  self.quaternion = normalize_quaternion(self.quaternion)
tensor([[0.2441],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.1431],
        [0.0000],
        [0.0230]], device='cuda:0')
Collision Distance:
World: tensor([0.0000, 0.0000, 0.0000, 0.0000, 1.8775], device='cuda:0')
Self: tensor([0., 0., 0., 0., 0.], device='cuda:0')
(curobo-env) ➜  curobo git:(main) python examples/ik_example.py
Traceback (most recent call last):
  File "examples/ik_example.py", line 232, in <module>
    demo_basic_ik()
  File "examples/ik_example.py", line 42, in demo_basic_ik
    ik_config = IKSolverConfig.load_from_robot_config(
  File "/home/pren/anaconda3/envs/curobo-env/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/wrap/reacher/ik_solver.py", line 203, in load_from_robot_config
    parallel_mppi = ParallelMPPI(mppi_cfg)
  File "/home/pren/anaconda3/envs/curobo-env/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/opt/particle/parallel_mppi.py", line 156, in __init__
    self.update_samples()
  File "/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/opt/particle/parallel_mppi.py", line 528, in update_samples
    self.sample_lib.get_samples(
  File "/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/util/sample_lib.py", line 378, in get_samples
    samples = self.sample_fns[k](sample_shape=s_shape)
  File "/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/util/sample_lib.py", line 125, in get_samples
    self.samples = self.halton_generator.get_gaussian_samples(sample_shape[0])
  File "/home/pren/anaconda3/envs/curobo-env/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/util/sample_lib.py", line 503, in get_gaussian_samples
    gaussian_halton_samples = gaussian_transform(
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
  File "/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/util/sample_lib.py", line 527, in gaussian_transform
    # these values.
    changed_samples = 1.99 * uniform_samples - 0.99
    gaussian_halton_samples = proj_mat * torch.erfinv(changed_samples)
                                         ~~~~~~~~~~~~ <--- HERE
    i_mat = i_mat * variance
    gaussian_halton_samples = torch.matmul(gaussian_halton_samples, i_mat)
RuntimeError: 
  #define POS_INFINITY __int_as_float(0x7f800000)
  #define INFINITY POS_INFINITY
  #define NEG_INFINITY __int_as_float(0xff800000)
  #define NAN __int_as_float(0x7fffffff)

  typedef long long int int64_t;
  typedef unsigned int uint32_t;
  typedef signed char int8_t;
  typedef unsigned char uint8_t;  // NOTE: this MUST be "unsigned char"! "char" is equivalent to "signed char"
  typedef short int16_t;
  static_assert(sizeof(int64_t) == 8, "expected size does not match");
  static_assert(sizeof(uint32_t) == 4, "expected size does not match");
  static_assert(sizeof(int8_t) == 1, "expected size does not match");
  constexpr int num_threads = 128;
  constexpr int thread_work_size = 4; // TODO: make template substitution once we decide where those vars live
  constexpr int block_work_size = thread_work_size * num_threads;
  //TODO use _assert_fail, because assert is disabled in non-debug builds
  #define ERROR_UNSUPPORTED_CAST assert(false);

  namespace std {

  using ::signbit;
  using ::isfinite;
  using ::isinf;
  using ::isnan;

  using ::abs;

  using ::acos;
  using ::acosf;
  using ::asin;
  using ::asinf;
  using ::atan;
  using ::atanf;
  using ::atan2;
  using ::atan2f;
  using ::ceil;
  using ::ceilf;
  using ::cos;
  using ::cosf;
  using ::cosh;
  using ::coshf;

  using ::exp;
  using ::expf;

  using ::fabs;
  using ::fabsf;
  using ::floor;
  using ::floorf;

  using ::fmod;
  using ::fmodf;

  using ::frexp;
  using ::frexpf;
  using ::ldexp;
  using ::ldexpf;

  using ::log;
  using ::logf;

  using ::log10;
  using ::log10f;
  using ::modf;
  using ::modff;

  using ::pow;
  using ::powf;

  using ::sin;
  using ::sinf;
  using ::sinh;
  using ::sinhf;

  using ::sqrt;
  using ::sqrtf;
  using ::tan;
  using ::tanf;

  using ::tanh;
  using ::tanhf;

  using ::acosh;
  using ::acoshf;
  using ::asinh;
  using ::asinhf;
  using ::atanh;
  using ::atanhf;
  using ::cbrt;
  using ::cbrtf;

  using ::copysign;
  using ::copysignf;

  using ::erf;
  using ::erff;
  using ::erfc;
  using ::erfcf;
  using ::exp2;
  using ::exp2f;
  using ::expm1;
  using ::expm1f;
  using ::fdim;
  using ::fdimf;
  using ::fmaf;
  using ::fma;
  using ::fmax;
  using ::fmaxf;
  using ::fmin;
  using ::fminf;
  using ::hypot;
  using ::hypotf;
  using ::ilogb;
  using ::ilogbf;
  using ::lgamma;
  using ::lgammaf;
  using ::llrint;
  using ::llrintf;
  using ::llround;
  using ::llroundf;
  using ::log1p;
  using ::log1pf;
  using ::log2;
  using ::log2f;
  using ::logb;
  using ::logbf;
  using ::lrint;
  using ::lrintf;
  using ::lround;
  using ::lroundf;

  using ::nan;
  using ::nanf;

  using ::nearbyint;
  using ::nearbyintf;
  using ::nextafter;
  using ::nextafterf;
  using ::remainder;
  using ::remainderf;
  using ::remquo;
  using ::remquof;
  using ::rint;
  using ::rintf;
  using ::round;
  using ::roundf;
  using ::scalbln;
  using ::scalblnf;
  using ::scalbn;
  using ::scalbnf;
  using ::tgamma;
  using ::tgammaf;
  using ::trunc;
  using ::truncf;

  } // namespace std

  // NB: Order matters for this macro; it is relied upon in
  // _promoteTypesLookup and the serialization format.
  // Note, some types have ctype as void because we don't support them in codegen
  #define AT_FORALL_SCALAR_TYPES_WITH_COMPLEX(_) \
  _(uint8_t, Byte) /* 0 */                               \
  _(int8_t, Char) /* 1 */                                \
  _(int16_t, Short) /* 2 */                              \
  _(int, Int) /* 3 */                                    \
  _(int64_t, Long) /* 4 */                               \
  _(at::Half, Half) /* 5 */                                  \
  _(float, Float) /* 6 */                                \
  _(double, Double) /* 7 */                              \
  _(std::complex<at::Half>, ComplexHalf) /* 8 */        \
  _(std::complex<float>, ComplexFloat) /* 9 */                          \
  _(std::complex<double>, ComplexDouble) /* 10 */                         \
  _(bool, Bool) /* 11 */                                 \
  _(void, QInt8) /* 12 */                          \
  _(void, QUInt8) /* 13 */                        \
  _(void, QInt32) /* 14 */                        \
  _(at::BFloat16, BFloat16) /* 15 */                             \

  #define AT_FORALL_SCALAR_TYPES_WITH_COMPLEX_EXCEPT_QINT(_)       \
  _(uint8_t, Byte)                                                 \
  _(int8_t, Char)                                                  \
  _(int16_t, Short)                                                \
  _(int, Int)                                                      \
  _(int64_t, Long)                                                 \
  _(at::Half, Half)                                                \
  _(float, Float)                                                  \
  _(double, Double)                                                \
  _(std::complex<at::Half>, ComplexHalf)                           \
  _(std::complex<float>, ComplexFloat)                             \
  _(std::complex<double>, ComplexDouble)                           \
  _(bool, Bool)                                                    \
  _(at::BFloat16, BFloat16)

  enum class ScalarType : int8_t {
  #define DEFINE_ENUM(_1, n) n,
  AT_FORALL_SCALAR_TYPES_WITH_COMPLEX(DEFINE_ENUM)
  #undef DEFINE_ENUM
      Undefined,
  NumOptions
  };

  template <typename T, int size>
  struct Array {
  T data[size];

  __device__ T operator[](int i) const {
      return data[i];
  }
  __device__ T& operator[](int i) {
      return data[i];
  }
  Array() = default;
  Array(const Array&) = default;
  Array& operator=(const Array&) = default;
  __device__ Array(T x) {
    for (int i = 0; i < size; i++) {
      data[i] = x;
    }
  }
  };

  template <typename scalar_t>
  __device__ __inline__ scalar_t load(char* base_ptr, uint32_t offset) {
      return *(reinterpret_cast<scalar_t*>(base_ptr) + offset);
  }

  template<typename scalar_t>
  __device__ __inline__ void store(scalar_t value, char *base_ptr, uint32_t offset) {
      *(reinterpret_cast<scalar_t *>(base_ptr) + offset) = value;
  }

  // aligned vector generates vectorized load/store on CUDA
  template<typename scalar_t, int vec_size>
  struct alignas(sizeof(scalar_t) * vec_size) aligned_vector {
    scalar_t val[vec_size];
  };

  template <typename T> T erfinv_kernel(T a) { return erfinv(a); }

  // TODO: setup grid-stride loop

  extern "C" __global__
  void erfinv_kernel_vectorized4_kernel(
      const int N,
      Array<char*, 1+1> data,
      float scalar_val) //[1+1],
      {
      constexpr int vec_size = 4;
      int remaining = N - block_work_size * blockIdx.x;
      auto thread_idx = threadIdx.x;
      int idx = blockIdx.x;
      float arg0[4];

      float out0[4];

      if (remaining < block_work_size) {
        #pragma unroll
        for (int j = 0; j < thread_work_size; j++){
          if (thread_idx >= remaining) {
            break;
          }
          int linear_idx = thread_idx + block_work_size * idx;
          arg0[j] = load<float>(data[1], linear_idx);

          thread_idx += num_threads;
        }
        #pragma unroll
        for (int j = 0; j < thread_work_size; j++) {
          if ((threadIdx.x  + j*num_threads) < remaining) {
            out0[j] = erfinv_kernel<float>(arg0[j] );
          }
        }
        thread_idx = threadIdx.x;
        #pragma unroll
        for (int j = 0; j < thread_work_size; j++) {
          if (thread_idx >= remaining) {
              break;
          }
          int linear_idx = thread_idx + block_work_size * idx;
          store<float>(out0[j], data[0], linear_idx);

          thread_idx += num_threads;
        }
      } else {
        static constexpr int loop_size = thread_work_size / vec_size;
  //actual loading
        using vec_t_input = aligned_vector<float, vec_size>;
        vec_t_input * vec0 = reinterpret_cast<vec_t_input *>(data[0+1]) + block_work_size / vec_size * idx;

        #pragma unroll
        for (int i = 0; i<loop_size; i++){
          vec_t_input v;
          v = vec0[thread_idx];
          #pragma unroll
          for (int j=0; j < vec_size; j++){
            arg0[vec_size * i + j] = v.val[j];
          }

          thread_idx += num_threads;
        }

        #pragma unroll
        for (int j = 0; j < thread_work_size; j++) {
          out0[j] = erfinv_kernel<float>(arg0[j] );
        }

        using vec_t_output = aligned_vector<float, vec_size>;
        vec_t_output* to_0 = reinterpret_cast<vec_t_output*>(data[0]) + block_work_size / vec_size * idx;

        int thread_idx = threadIdx.x;
        #pragma unroll
        for (int i = 0; i<loop_size; i++){
          vec_t_output v;
          #pragma unroll
          for (int j=0; j<vec_size; j++){
          v.val[j] = out0[vec_size * i + j];
          }
          to_0[thread_idx] = v;

          thread_idx += num_threads;
        }
      }
  }
nvrtc: error: invalid value for --gpu-architecture (-arch)

(curobo-env) ➜  curobo git:(main) python3 -m pytest .
Traceback (most recent call last):
  File "/home/pren/anaconda3/envs/curobo-env/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/pren/anaconda3/envs/curobo-env/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/pren/anaconda3/envs/curobo-env/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/home/pren/.local/lib/python3.8/site-packages/pytest/__init__.py", line 5, in <module>
    from _pytest._code import ExceptionInfo
  File "/home/pren/.local/lib/python3.8/site-packages/_pytest/_code/__init__.py", line 2, in <module>
    from .code import Code
  File "/home/pren/.local/lib/python3.8/site-packages/_pytest/_code/code.py", line 36, in <module>
    import pluggy
ModuleNotFoundError: No module named 'pluggy'
(curobo-env) ➜  curobo git:(main) pip install pluggy
Looking in indexes: https://mirrors.bfsu.edu.cn/pypi/web/simple/
Collecting pluggy
  Using cached https://mirrors.bfsu.edu.cn/pypi/web/packages/05/b8/42ed91898d4784546c5f06c60506400548db3f7a4b3fb441cba4e5c17952/pluggy-1.3.0-py3-none-any.whl (18 kB)
Installing collected packages: pluggy
Successfully installed pluggy-1.3.0

[notice] A new release of pip is available: 23.3.1 -> 23.3.2
[notice] To update, run: pip install --upgrade pip
(curobo-env) ➜  curobo git:(main) python3 -m pytest .
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.3.0
rootdir: /home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo, configfile: pyproject.toml
plugins: anyio-4.0.0, torchtyping-0.1.4, typeguard-4.1.5, cov-3.0.0, hydra-core-1.3.2
collected 129 items / 2 errors / 2 skipped                                     

==================================== ERRORS ====================================
___________________ ERROR collecting tests/ik_config_test.py ___________________
tests/ik_config_test.py:134: in <module>
    (ik_base_config(), True),
tests/ik_config_test.py:39: in ik_base_config
    ik_config = IKSolverConfig.load_from_robot_config(
/home/pren/anaconda3/envs/curobo-env/lib/python3.8/contextlib.py:75: in inner
    return func(*args, **kwds)
src/curobo/wrap/reacher/ik_solver.py:203: in load_from_robot_config
    parallel_mppi = ParallelMPPI(mppi_cfg)
/home/pren/anaconda3/envs/curobo-env/lib/python3.8/contextlib.py:75: in inner
    return func(*args, **kwds)
src/curobo/opt/particle/parallel_mppi.py:156: in __init__
    self.update_samples()
src/curobo/opt/particle/parallel_mppi.py:528: in update_samples
    self.sample_lib.get_samples(
src/curobo/util/sample_lib.py:378: in get_samples
    samples = self.sample_fns[k](sample_shape=s_shape)
src/curobo/util/sample_lib.py:125: in get_samples
    self.samples = self.halton_generator.get_gaussian_samples(sample_shape[0])
/home/pren/anaconda3/envs/curobo-env/lib/python3.8/contextlib.py:75: in inner
    return func(*args, **kwds)
src/curobo/util/sample_lib.py:503: in get_gaussian_samples
    gaussian_halton_samples = gaussian_transform(
E   RuntimeError: The following operation failed in the TorchScript interpreter.
E   Traceback of TorchScript (most recent call last):
E     File "/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/util/sample_lib.py", line 527, in gaussian_transform
E       # these values.
E       changed_samples = 1.99 * uniform_samples - 0.99
E       gaussian_halton_samples = proj_mat * torch.erfinv(changed_samples)
E                                            ~~~~~~~~~~~~ <--- HERE
E       i_mat = i_mat * variance
E       gaussian_halton_samples = torch.matmul(gaussian_halton_samples, i_mat)
E   RuntimeError: 
E     #define POS_INFINITY __int_as_float(0x7f800000)
E     #define INFINITY POS_INFINITY
E     #define NEG_INFINITY __int_as_float(0xff800000)
E     #define NAN __int_as_float(0x7fffffff)
E   
E     typedef long long int int64_t;
E     typedef unsigned int uint32_t;
E     typedef signed char int8_t;
E     typedef unsigned char uint8_t;  // NOTE: this MUST be "unsigned char"! "char" is equivalent to "signed char"
E     typedef short int16_t;
E     static_assert(sizeof(int64_t) == 8, "expected size does not match");
E     static_assert(sizeof(uint32_t) == 4, "expected size does not match");
E     static_assert(sizeof(int8_t) == 1, "expected size does not match");
E     constexpr int num_threads = 128;
E     constexpr int thread_work_size = 4; // TODO: make template substitution once we decide where those vars live
E     constexpr int block_work_size = thread_work_size * num_threads;
E     //TODO use _assert_fail, because assert is disabled in non-debug builds
E     #define ERROR_UNSUPPORTED_CAST assert(false);
E   
E     
E     
E     
E     namespace std {
E     
E     using ::signbit;
E     using ::isfinite;
E     using ::isinf;
E     using ::isnan;
E     
E     using ::abs;
E     
E     using ::acos;
E     using ::acosf;
E     using ::asin;
E     using ::asinf;
E     using ::atan;
E     using ::atanf;
E     using ::atan2;
E     using ::atan2f;
E     using ::ceil;
E     using ::ceilf;
E     using ::cos;
E     using ::cosf;
E     using ::cosh;
E     using ::coshf;
E     
E     using ::exp;
E     using ::expf;
E     
E     using ::fabs;
E     using ::fabsf;
E     using ::floor;
E     using ::floorf;
E     
E     using ::fmod;
E     using ::fmodf;
E     
E     using ::frexp;
E     using ::frexpf;
E     using ::ldexp;
E     using ::ldexpf;
E     
E     using ::log;
E     using ::logf;
E     
E     using ::log10;
E     using ::log10f;
E     using ::modf;
E     using ::modff;
E     
E     using ::pow;
E     using ::powf;
E     
E     using ::sin;
E     using ::sinf;
E     using ::sinh;
E     using ::sinhf;
E     
E     using ::sqrt;
E     using ::sqrtf;
E     using ::tan;
E     using ::tanf;
E     
E     using ::tanh;
E     using ::tanhf;
E     
E     using ::acosh;
E     using ::acoshf;
E     using ::asinh;
E     using ::asinhf;
E     using ::atanh;
E     using ::atanhf;
E     using ::cbrt;
E     using ::cbrtf;
E     
E     using ::copysign;
E     using ::copysignf;
E     
E     using ::erf;
E     using ::erff;
E     using ::erfc;
E     using ::erfcf;
E     using ::exp2;
E     using ::exp2f;
E     using ::expm1;
E     using ::expm1f;
E     using ::fdim;
E     using ::fdimf;
E     using ::fmaf;
E     using ::fma;
E     using ::fmax;
E     using ::fmaxf;
E     using ::fmin;
E     using ::fminf;
E     using ::hypot;
E     using ::hypotf;
E     using ::ilogb;
E     using ::ilogbf;
E     using ::lgamma;
E     using ::lgammaf;
E     using ::llrint;
E     using ::llrintf;
E     using ::llround;
E     using ::llroundf;
E     using ::log1p;
E     using ::log1pf;
E     using ::log2;
E     using ::log2f;
E     using ::logb;
E     using ::logbf;
E     using ::lrint;
E     using ::lrintf;
E     using ::lround;
E     using ::lroundf;
E     
E     using ::nan;
E     using ::nanf;
E     
E     using ::nearbyint;
E     using ::nearbyintf;
E     using ::nextafter;
E     using ::nextafterf;
E     using ::remainder;
E     using ::remainderf;
E     using ::remquo;
E     using ::remquof;
E     using ::rint;
E     using ::rintf;
E     using ::round;
E     using ::roundf;
E     using ::scalbln;
E     using ::scalblnf;
E     using ::scalbn;
E     using ::scalbnf;
E     using ::tgamma;
E     using ::tgammaf;
E     using ::trunc;
E     using ::truncf;
E     
E     } // namespace std
E     
E     
E   
E     // NB: Order matters for this macro; it is relied upon in
E     // _promoteTypesLookup and the serialization format.
E     // Note, some types have ctype as void because we don't support them in codegen
E     #define AT_FORALL_SCALAR_TYPES_WITH_COMPLEX(_) \
E     _(uint8_t, Byte) /* 0 */                               \
E     _(int8_t, Char) /* 1 */                                \
E     _(int16_t, Short) /* 2 */                              \
E     _(int, Int) /* 3 */                                    \
E     _(int64_t, Long) /* 4 */                               \
E     _(at::Half, Half) /* 5 */                                  \
E     _(float, Float) /* 6 */                                \
E     _(double, Double) /* 7 */                              \
E     _(std::complex<at::Half>, ComplexHalf) /* 8 */        \
E     _(std::complex<float>, ComplexFloat) /* 9 */                          \
E     _(std::complex<double>, ComplexDouble) /* 10 */                         \
E     _(bool, Bool) /* 11 */                                 \
E     _(void, QInt8) /* 12 */                          \
E     _(void, QUInt8) /* 13 */                        \
E     _(void, QInt32) /* 14 */                        \
E     _(at::BFloat16, BFloat16) /* 15 */                             \
E   
E     #define AT_FORALL_SCALAR_TYPES_WITH_COMPLEX_EXCEPT_QINT(_)       \
E     _(uint8_t, Byte)                                                 \
E     _(int8_t, Char)                                                  \
E     _(int16_t, Short)                                                \
E     _(int, Int)                                                      \
E     _(int64_t, Long)                                                 \
E     _(at::Half, Half)                                                \
E     _(float, Float)                                                  \
E     _(double, Double)                                                \
E     _(std::complex<at::Half>, ComplexHalf)                           \
E     _(std::complex<float>, ComplexFloat)                             \
E     _(std::complex<double>, ComplexDouble)                           \
E     _(bool, Bool)                                                    \
E     _(at::BFloat16, BFloat16)
E   
E   
E     enum class ScalarType : int8_t {
E     #define DEFINE_ENUM(_1, n) n,
E     AT_FORALL_SCALAR_TYPES_WITH_COMPLEX(DEFINE_ENUM)
E     #undef DEFINE_ENUM
E         Undefined,
E     NumOptions
E     };
E   
E     template <typename T, int size>
E     struct Array {
E     T data[size];
E   
E     __device__ T operator[](int i) const {
E         return data[i];
E     }
E     __device__ T& operator[](int i) {
E         return data[i];
E     }
E     Array() = default;
E     Array(const Array&) = default;
E     Array& operator=(const Array&) = default;
E     __device__ Array(T x) {
E       for (int i = 0; i < size; i++) {
E         data[i] = x;
E       }
E     }
E     };
E   
E     
E     
E     
E     
E     
E   
E   
E   
E   
E     template <typename scalar_t>
E     __device__ __inline__ scalar_t load(char* base_ptr, uint32_t offset) {
E         return *(reinterpret_cast<scalar_t*>(base_ptr) + offset);
E     }
E   
E     template<typename scalar_t>
E     __device__ __inline__ void store(scalar_t value, char *base_ptr, uint32_t offset) {
E         *(reinterpret_cast<scalar_t *>(base_ptr) + offset) = value;
E     }
E   
E     // aligned vector generates vectorized load/store on CUDA
E     template<typename scalar_t, int vec_size>
E     struct alignas(sizeof(scalar_t) * vec_size) aligned_vector {
E       scalar_t val[vec_size];
E     };
E   
E     template <typename T> T erfinv_kernel(T a) { return erfinv(a); }
E   
E     // TODO: setup grid-stride loop
E   
E     extern "C" __global__
E     void erfinv_kernel_vectorized4_kernel(
E         const int N,
E         Array<char*, 1+1> data,
E         float scalar_val) //[1+1],
E         {
E         constexpr int vec_size = 4;
E         int remaining = N - block_work_size * blockIdx.x;
E         auto thread_idx = threadIdx.x;
E         int idx = blockIdx.x;
E         float arg0[4];
E         
E         float out0[4];
E         
E   
E         if (remaining < block_work_size) {
E           #pragma unroll
E           for (int j = 0; j < thread_work_size; j++){
E             if (thread_idx >= remaining) {
E               break;
E             }
E             int linear_idx = thread_idx + block_work_size * idx;
E             arg0[j] = load<float>(data[1], linear_idx);
E             
E             thread_idx += num_threads;
E           }
E           #pragma unroll
E           for (int j = 0; j < thread_work_size; j++) {
E             if ((threadIdx.x  + j*num_threads) < remaining) {
E               out0[j] = erfinv_kernel<float>(arg0[j] );
E             }
E           }
E           thread_idx = threadIdx.x;
E           #pragma unroll
E           for (int j = 0; j < thread_work_size; j++) {
E             if (thread_idx >= remaining) {
E                 break;
E             }
E             int linear_idx = thread_idx + block_work_size * idx;
E             store<float>(out0[j], data[0], linear_idx);
E             
E             thread_idx += num_threads;
E           }
E         } else {
E           static constexpr int loop_size = thread_work_size / vec_size;
E     //actual loading
E           using vec_t_input = aligned_vector<float, vec_size>;
E           vec_t_input * vec0 = reinterpret_cast<vec_t_input *>(data[0+1]) + block_work_size / vec_size * idx;
E           
E           #pragma unroll
E           for (int i = 0; i<loop_size; i++){
E             vec_t_input v;
E             v = vec0[thread_idx];
E             #pragma unroll
E             for (int j=0; j < vec_size; j++){
E               arg0[vec_size * i + j] = v.val[j];
E             }
E             
E             thread_idx += num_threads;
E           }
E   
E           #pragma unroll
E           for (int j = 0; j < thread_work_size; j++) {
E             out0[j] = erfinv_kernel<float>(arg0[j] );
E           }
E   
E           using vec_t_output = aligned_vector<float, vec_size>;
E           vec_t_output* to_0 = reinterpret_cast<vec_t_output*>(data[0]) + block_work_size / vec_size * idx;
E           
E           int thread_idx = threadIdx.x;
E           #pragma unroll
E           for (int i = 0; i<loop_size; i++){
E             vec_t_output v;
E             #pragma unroll
E             for (int j=0; j<vec_size; j++){
E             v.val[j] = out0[vec_size * i + j];
E             }
E             to_0[thread_idx] = v;
E             
E             thread_idx += num_threads;
E           }
E         }
E     }
E   nvrtc: error: invalid value for --gpu-architecture (-arch)
________________ ERROR collecting tests/trajopt_config_test.py _________________
tests/trajopt_config_test.py:115: in <module>
    (trajopt_base_config(), True),
tests/trajopt_config_test.py:35: in trajopt_base_config
    trajopt_config = TrajOptSolverConfig.load_from_robot_config(
/home/pren/anaconda3/envs/curobo-env/lib/python3.8/contextlib.py:75: in inner
    return func(*args, **kwds)
src/curobo/wrap/reacher/trajopt.py:284: in load_from_robot_config
    parallel_mppi = ParallelMPPI(mppi_cfg)
/home/pren/anaconda3/envs/curobo-env/lib/python3.8/contextlib.py:75: in inner
    return func(*args, **kwds)
src/curobo/opt/particle/parallel_mppi.py:155: in __init__
    self.reset_distribution()
src/curobo/opt/particle/parallel_mppi.py:171: in reset_distribution
    self.reset_mean()
src/curobo/opt/particle/parallel_mppi.py:356: in reset_mean
    mean = self.mean_lib.get_samples([self.n_envs])
src/curobo/util/sample_lib.py:125: in get_samples
    self.samples = self.halton_generator.get_gaussian_samples(sample_shape[0])
/home/pren/anaconda3/envs/curobo-env/lib/python3.8/contextlib.py:75: in inner
    return func(*args, **kwds)
src/curobo/util/sample_lib.py:503: in get_gaussian_samples
    gaussian_halton_samples = gaussian_transform(
E   RuntimeError: The following operation failed in the TorchScript interpreter.
E   Traceback of TorchScript (most recent call last):
E     File "/home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/util/sample_lib.py", line 527, in gaussian_transform
E       # these values.
E       changed_samples = 1.99 * uniform_samples - 0.99
E       gaussian_halton_samples = proj_mat * torch.erfinv(changed_samples)
E                                            ~~~~~~~~~~~~ <--- HERE
E       i_mat = i_mat * variance
E       gaussian_halton_samples = torch.matmul(gaussian_halton_samples, i_mat)
E   RuntimeError: 
E     #define POS_INFINITY __int_as_float(0x7f800000)
E     #define INFINITY POS_INFINITY
E     #define NEG_INFINITY __int_as_float(0xff800000)
E     #define NAN __int_as_float(0x7fffffff)
E   
E     typedef long long int int64_t;
E     typedef unsigned int uint32_t;
E     typedef signed char int8_t;
E     typedef unsigned char uint8_t;  // NOTE: this MUST be "unsigned char"! "char" is equivalent to "signed char"
E     typedef short int16_t;
E     static_assert(sizeof(int64_t) == 8, "expected size does not match");
E     static_assert(sizeof(uint32_t) == 4, "expected size does not match");
E     static_assert(sizeof(int8_t) == 1, "expected size does not match");
E     constexpr int num_threads = 128;
E     constexpr int thread_work_size = 4; // TODO: make template substitution once we decide where those vars live
E     constexpr int block_work_size = thread_work_size * num_threads;
E     //TODO use _assert_fail, because assert is disabled in non-debug builds
E     #define ERROR_UNSUPPORTED_CAST assert(false);
E   
E     
E     
E     
E     namespace std {
E     
E     using ::signbit;
E     using ::isfinite;
E     using ::isinf;
E     using ::isnan;
E     
E     using ::abs;
E     
E     using ::acos;
E     using ::acosf;
E     using ::asin;
E     using ::asinf;
E     using ::atan;
E     using ::atanf;
E     using ::atan2;
E     using ::atan2f;
E     using ::ceil;
E     using ::ceilf;
E     using ::cos;
E     using ::cosf;
E     using ::cosh;
E     using ::coshf;
E     
E     using ::exp;
E     using ::expf;
E     
E     using ::fabs;
E     using ::fabsf;
E     using ::floor;
E     using ::floorf;
E     
E     using ::fmod;
E     using ::fmodf;
E     
E     using ::frexp;
E     using ::frexpf;
E     using ::ldexp;
E     using ::ldexpf;
E     
E     using ::log;
E     using ::logf;
E     
E     using ::log10;
E     using ::log10f;
E     using ::modf;
E     using ::modff;
E     
E     using ::pow;
E     using ::powf;
E     
E     using ::sin;
E     using ::sinf;
E     using ::sinh;
E     using ::sinhf;
E     
E     using ::sqrt;
E     using ::sqrtf;
E     using ::tan;
E     using ::tanf;
E     
E     using ::tanh;
E     using ::tanhf;
E     
E     using ::acosh;
E     using ::acoshf;
E     using ::asinh;
E     using ::asinhf;
E     using ::atanh;
E     using ::atanhf;
E     using ::cbrt;
E     using ::cbrtf;
E     
E     using ::copysign;
E     using ::copysignf;
E     
E     using ::erf;
E     using ::erff;
E     using ::erfc;
E     using ::erfcf;
E     using ::exp2;
E     using ::exp2f;
E     using ::expm1;
E     using ::expm1f;
E     using ::fdim;
E     using ::fdimf;
E     using ::fmaf;
E     using ::fma;
E     using ::fmax;
E     using ::fmaxf;
E     using ::fmin;
E     using ::fminf;
E     using ::hypot;
E     using ::hypotf;
E     using ::ilogb;
E     using ::ilogbf;
E     using ::lgamma;
E     using ::lgammaf;
E     using ::llrint;
E     using ::llrintf;
E     using ::llround;
E     using ::llroundf;
E     using ::log1p;
E     using ::log1pf;
E     using ::log2;
E     using ::log2f;
E     using ::logb;
E     using ::logbf;
E     using ::lrint;
E     using ::lrintf;
E     using ::lround;
E     using ::lroundf;
E     
E     using ::nan;
E     using ::nanf;
E     
E     using ::nearbyint;
E     using ::nearbyintf;
E     using ::nextafter;
E     using ::nextafterf;
E     using ::remainder;
E     using ::remainderf;
E     using ::remquo;
E     using ::remquof;
E     using ::rint;
E     using ::rintf;
E     using ::round;
E     using ::roundf;
E     using ::scalbln;
E     using ::scalblnf;
E     using ::scalbn;
E     using ::scalbnf;
E     using ::tgamma;
E     using ::tgammaf;
E     using ::trunc;
E     using ::truncf;
E     
E     } // namespace std
E     
E     
E   
E     // NB: Order matters for this macro; it is relied upon in
E     // _promoteTypesLookup and the serialization format.
E     // Note, some types have ctype as void because we don't support them in codegen
E     #define AT_FORALL_SCALAR_TYPES_WITH_COMPLEX(_) \
E     _(uint8_t, Byte) /* 0 */                               \
E     _(int8_t, Char) /* 1 */                                \
E     _(int16_t, Short) /* 2 */                              \
E     _(int, Int) /* 3 */                                    \
E     _(int64_t, Long) /* 4 */                               \
E     _(at::Half, Half) /* 5 */                                  \
E     _(float, Float) /* 6 */                                \
E     _(double, Double) /* 7 */                              \
E     _(std::complex<at::Half>, ComplexHalf) /* 8 */        \
E     _(std::complex<float>, ComplexFloat) /* 9 */                          \
E     _(std::complex<double>, ComplexDouble) /* 10 */                         \
E     _(bool, Bool) /* 11 */                                 \
E     _(void, QInt8) /* 12 */                          \
E     _(void, QUInt8) /* 13 */                        \
E     _(void, QInt32) /* 14 */                        \
E     _(at::BFloat16, BFloat16) /* 15 */                             \
E   
E     #define AT_FORALL_SCALAR_TYPES_WITH_COMPLEX_EXCEPT_QINT(_)       \
E     _(uint8_t, Byte)                                                 \
E     _(int8_t, Char)                                                  \
E     _(int16_t, Short)                                                \
E     _(int, Int)                                                      \
E     _(int64_t, Long)                                                 \
E     _(at::Half, Half)                                                \
E     _(float, Float)                                                  \
E     _(double, Double)                                                \
E     _(std::complex<at::Half>, ComplexHalf)                           \
E     _(std::complex<float>, ComplexFloat)                             \
E     _(std::complex<double>, ComplexDouble)                           \
E     _(bool, Bool)                                                    \
E     _(at::BFloat16, BFloat16)
E   
E   
E     enum class ScalarType : int8_t {
E     #define DEFINE_ENUM(_1, n) n,
E     AT_FORALL_SCALAR_TYPES_WITH_COMPLEX(DEFINE_ENUM)
E     #undef DEFINE_ENUM
E         Undefined,
E     NumOptions
E     };
E   
E     template <typename T, int size>
E     struct Array {
E     T data[size];
E   
E     __device__ T operator[](int i) const {
E         return data[i];
E     }
E     __device__ T& operator[](int i) {
E         return data[i];
E     }
E     Array() = default;
E     Array(const Array&) = default;
E     Array& operator=(const Array&) = default;
E     __device__ Array(T x) {
E       for (int i = 0; i < size; i++) {
E         data[i] = x;
E       }
E     }
E     };
E   
E     
E     
E     
E     
E     
E   
E   
E   
E   
E     template <typename scalar_t>
E     __device__ __inline__ scalar_t load(char* base_ptr, uint32_t offset) {
E         return *(reinterpret_cast<scalar_t*>(base_ptr) + offset);
E     }
E   
E     template<typename scalar_t>
E     __device__ __inline__ void store(scalar_t value, char *base_ptr, uint32_t offset) {
E         *(reinterpret_cast<scalar_t *>(base_ptr) + offset) = value;
E     }
E   
E     // aligned vector generates vectorized load/store on CUDA
E     template<typename scalar_t, int vec_size>
E     struct alignas(sizeof(scalar_t) * vec_size) aligned_vector {
E       scalar_t val[vec_size];
E     };
E   
E     template <typename T> T erfinv_kernel(T a) { return erfinv(a); }
E   
E     // TODO: setup grid-stride loop
E   
E     extern "C" __global__
E     void erfinv_kernel_vectorized4_kernel(
E         const int N,
E         Array<char*, 1+1> data,
E         float scalar_val) //[1+1],
E         {
E         constexpr int vec_size = 4;
E         int remaining = N - block_work_size * blockIdx.x;
E         auto thread_idx = threadIdx.x;
E         int idx = blockIdx.x;
E         float arg0[4];
E         
E         float out0[4];
E         
E   
E         if (remaining < block_work_size) {
E           #pragma unroll
E           for (int j = 0; j < thread_work_size; j++){
E             if (thread_idx >= remaining) {
E               break;
E             }
E             int linear_idx = thread_idx + block_work_size * idx;
E             arg0[j] = load<float>(data[1], linear_idx);
E             
E             thread_idx += num_threads;
E           }
E           #pragma unroll
E           for (int j = 0; j < thread_work_size; j++) {
E             if ((threadIdx.x  + j*num_threads) < remaining) {
E               out0[j] = erfinv_kernel<float>(arg0[j] );
E             }
E           }
E           thread_idx = threadIdx.x;
E           #pragma unroll
E           for (int j = 0; j < thread_work_size; j++) {
E             if (thread_idx >= remaining) {
E                 break;
E             }
E             int linear_idx = thread_idx + block_work_size * idx;
E             store<float>(out0[j], data[0], linear_idx);
E             
E             thread_idx += num_threads;
E           }
E         } else {
E           static constexpr int loop_size = thread_work_size / vec_size;
E     //actual loading
E           using vec_t_input = aligned_vector<float, vec_size>;
E           vec_t_input * vec0 = reinterpret_cast<vec_t_input *>(data[0+1]) + block_work_size / vec_size * idx;
E           
E           #pragma unroll
E           for (int i = 0; i<loop_size; i++){
E             vec_t_input v;
E             v = vec0[thread_idx];
E             #pragma unroll
E             for (int j=0; j < vec_size; j++){
E               arg0[vec_size * i + j] = v.val[j];
E             }
E             
E             thread_idx += num_threads;
E           }
E   
E           #pragma unroll
E           for (int j = 0; j < thread_work_size; j++) {
E             out0[j] = erfinv_kernel<float>(arg0[j] );
E           }
E   
E           using vec_t_output = aligned_vector<float, vec_size>;
E           vec_t_output* to_0 = reinterpret_cast<vec_t_output*>(data[0]) + block_work_size / vec_size * idx;
E           
E           int thread_idx = threadIdx.x;
E           #pragma unroll
E           for (int i = 0; i<loop_size; i++){
E             vec_t_output v;
E             #pragma unroll
E             for (int j=0; j<vec_size; j++){
E             v.val[j] = out0[vec_size * i + j];
E             }
E             to_0[thread_idx] = v;
E             
E             thread_idx += num_threads;
E           }
E         }
E     }
E   nvrtc: error: invalid value for --gpu-architecture (-arch)
=============================== warnings summary ===============================
<unknown>:3
  <unknown>:3: DeprecationWarning: invalid escape sequence \p

tests/ik_config_test.py:149
  /home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/tests/ik_config_test.py:149: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    if expected is not -100:

src/curobo/types/math.py:83
src/curobo/types/math.py:83
  /home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/types/math.py:83: UserWarning: FALLBACK path has been taken inside: runCudaFusionGroup. This is an indication that codegen Failed for some reason.
  To debug try disable codegen fallback path via setting the env variable `export PYTORCH_NVFUSER_DISABLE=fallback`
   (Triggered internally at  ../torch/csrc/jit/codegen/cuda/manager.cpp:329.)
    self.quaternion = normalize_quaternion(self.quaternion)

src/curobo/opt/particle/particle_opt_utils.py:67
  /home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/opt/particle/particle_opt_utils.py:67: UserWarning: FALLBACK path has been taken inside: runCudaFusionGroup. This is an indication that codegen Failed for some reason.
  To debug try disable codegen fallback path via setting the env variable `export PYTORCH_NVFUSER_DISABLE=fallback`
   (Triggered internally at  ../torch/csrc/jit/codegen/cuda/manager.cpp:329.)
    cov, scale_tril, scaled_M = get_stomp_cov_jit(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR tests/ik_config_test.py - RuntimeError: The following operation failed ...
ERROR tests/trajopt_config_test.py - RuntimeError: The following operation fa...
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
=================== 2 skipped, 5 warnings, 2 errors in 3.51s ===================
(curobo-env) ➜  curobo git:(main)

What gpu are you testing this on? Also what's your version of the nvidia driver?

What gpu are you testing this on? Also what's your version of the nvidia driver?

Hi Balakumar @balakumar-s , thx for your timely reply. I just update the above issue description. The GPU is RTX 4090 and the driver is 525.

➜  ~ nvidia-smi
Wed Dec 20 10:09:58 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0  On |                  Off |
| 32%   31C    P8    12W / 450W |   2567MiB / 24564MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1210      G   /usr/lib/xorg/Xorg                151MiB |
|    0   N/A  N/A      2038      G   /usr/lib/xorg/Xorg               1448MiB |
|    0   N/A  N/A      2167      G   /usr/bin/gnome-shell              257MiB |
|    0   N/A  N/A      2508      G   ...RendererForSitePerProcess       18MiB |
|    0   N/A  N/A      4024    C+G   ...369779272449050828,262144      595MiB |
|    0   N/A  N/A      5735      G   ...RendererForSitePerProcess       15MiB |
|    0   N/A  N/A     12806      G   ...RendererForSitePerProcess       60MiB |
+-----------------------------------------------------------------------------+
➜  ~ 
➜  ~ nvcc -V   
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_May__3_18:49:52_PDT_2022
Cuda compilation tools, release 11.7, V11.7.64
Build cuda_11.7.r11.7/compiler.31294372_0
➜  ~

Can you try with cuda 11.8?

Pytorch provides conda installation for cuda 11.8 here: https://pytorch.org/get-started/locally/

Looks like it's a known bug in pytorch 1.13 that was fixed.

https://github.com/pytorch/pytorch/issues/87595

Install the newer pytorch 2.0 and this should be fixed.

Can you try with cuda 11.8?

Pytorch provides conda installation for cuda 11.8 here: https://pytorch.org/get-started/locally/

I will try it now.

Hi Yibo @Spphire , I see that you can successfully run CuRobo demo in Isaac Sim. May I ask what your environment (Ubuntu, Python, CUDA, PyTorch, Isaac Sim) is?

Looks like it's a known bug in pytorch 1.13 that was fixed.

pytorch/pytorch#87595

Install the newer pytorch 2.0 and this should be fixed.

My setup is

conda create -n curobo-env python=3.8.10
conda activate curobo-env
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
conda install pip
mkdir -p /home/pren/Space/GitHub-Codes/curobo_ws/curobo
cd /home/pren/Space/GitHub-Codes/curobo_ws/curobo
git clone https://github.com/NVlabs/curobo.git
cd curobo
git lfs pull
pip install -e . --no-build-isolation
python examples/collision_check_example.py  # Passed
python examples/ik_example.py  # Passed
pip install pluggy
python3 -m pytest .  # 1 failed, 135 passed, 3 skipped, 5 warnings

➜  ~ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
➜  ~    
➜  ~ nvidia-smi
Wed Dec 20 14:33:51 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0  On |                  Off |
| 32%   34C    P8    12W / 450W |   1663MiB / 24564MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1210      G   /usr/lib/xorg/Xorg                151MiB |
|    0   N/A  N/A      2038      G   /usr/lib/xorg/Xorg                865MiB |
|    0   N/A  N/A      2167      G   /usr/bin/gnome-shell              134MiB |
|    0   N/A  N/A      2508      G   ...RendererForSitePerProcess       14MiB |
|    0   N/A  N/A      4024    C+G   ...369779272449050828,262144      414MiB |
|    0   N/A  N/A     12806      G   ...RendererForSitePerProcess       62MiB |
+-----------------------------------------------------------------------------+
➜  ~

Issue Details

(curobo-env) ➜  curobo git clone https://github.com/NVlabs/curobo.git
Cloning into 'curobo'...
remote: Enumerating objects: 854, done.
remote: Counting objects: 100% (74/74), done.
remote: Compressing objects: 100% (34/34), done.
remote: Total 854 (delta 52), reused 45 (delta 39), pack-reused 780
Receiving objects: 100% (854/854), 2.68 MiB | 1.52 MiB/s, done.
Resolving deltas: 100% (316/316), done.
Filtering content: 100% (214/214), 238.55 MiB | 1.15 MiB/s, done.
(curobo-env) ➜  curobo cd curobo
(curobo-env) ➜  curobo git:(main) git lfs pull
(curobo-env) ➜  curobo git:(main) pip install -e . --no-build-isolation
Looking in indexes: https://mirrors.bfsu.edu.cn/pypi/web/simple/
Obtaining file:///home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo
  Checking if build backend supports build_editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: networkx in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (3.1)
Requirement already satisfied: numpy in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (1.22.0)
Requirement already satisfied: numpy-quaternion in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (2022.4.3)
Requirement already satisfied: pyyaml in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (6.0.1)
Requirement already satisfied: setuptools-scm>=6.2 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (8.0.4)
Requirement already satisfied: torchtyping in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (0.1.4)
Requirement already satisfied: torch>=1.10 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (1.12.1+cu113)
Requirement already satisfied: trimesh in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (3.15.5)
Requirement already satisfied: yourdfpy>=0.0.53 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (0.0.53)
Requirement already satisfied: warp-lang>=0.9.0 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (0.10.1)
Requirement already satisfied: scipy>=1.7.0 in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (1.10.1)
Requirement already satisfied: tqdm in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (4.66.1)
Requirement already satisfied: wheel in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (0.41.2)
Requirement already satisfied: importlib-resources in /home/pren/.local/lib/python3.8/site-packages (from nvidia-curobo==0.6.2.post1.dev3) (6.1.0)
Requirement already satisfied: packaging>=20 in /home/pren/.local/lib/python3.8/site-packages (from setuptools-scm>=6.2->nvidia-curobo==0.6.2.post1.dev3) (23.2)
Requirement already satisfied: setuptools in /home/pren/.local/lib/python3.8/site-packages (from setuptools-scm>=6.2->nvidia-curobo==0.6.2.post1.dev3) (65.7.0)
Requirement already satisfied: typing-extensions in /home/pren/.local/lib/python3.8/site-packages (from setuptools-scm>=6.2->nvidia-curobo==0.6.2.post1.dev3) (4.8.0)
Requirement already satisfied: tomli>=1 in /home/pren/.local/lib/python3.8/site-packages (from setuptools-scm>=6.2->nvidia-curobo==0.6.2.post1.dev3) (2.0.1)
Collecting lxml (from yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3)
  Using cached https://mirrors.bfsu.edu.cn/pypi/web/packages/c3/4c/556867cb2193b323728678f239ce24463ae71b8636aba7d6e55a72e3650e/lxml-4.9.4-cp38-cp38-manylinux_2_28_x86_64.whl (8.0 MB)
Requirement already satisfied: six in /home/pren/anaconda3/envs/curobo-env/lib/python3.8/site-packages (from yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.16.0)
Requirement already satisfied: zipp>=3.1.0 in /home/pren/.local/lib/python3.8/site-packages (from importlib-resources->nvidia-curobo==0.6.2.post1.dev3) (3.17.0)
Requirement already satisfied: typeguard>=2.11.1 in /home/pren/.local/lib/python3.8/site-packages (from torchtyping->nvidia-curobo==0.6.2.post1.dev3) (4.1.5)
Requirement already satisfied: pillow in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (10.1.0)
Requirement already satisfied: requests in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2.31.0)
Collecting chardet (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3)
  Using cached https://mirrors.bfsu.edu.cn/pypi/web/packages/38/6f/f5fbc992a329ee4e0f288c1fe0e2ad9485ed064cac731ed2fe47dcc38cbf/chardet-5.2.0-py3-none-any.whl (199 kB)
Requirement already satisfied: msgpack in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.0.7)
Requirement already satisfied: xxhash in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (3.4.1)
Requirement already satisfied: shapely in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2.0.2)
Requirement already satisfied: sympy in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.12)
Requirement already satisfied: jsonschema in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (4.19.1)
Requirement already satisfied: mapbox-earcut in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.0.1)
Requirement already satisfied: pyglet in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.5.27)
Requirement already satisfied: rtree in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.1.0)
Requirement already satisfied: pycollada in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (0.7.2)
Requirement already satisfied: colorlog in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (6.7.0)
Requirement already satisfied: svg.path in /home/pren/.local/lib/python3.8/site-packages (from trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (6.3)
Requirement already satisfied: importlib-metadata>=3.6 in /home/pren/.local/lib/python3.8/site-packages (from typeguard>=2.11.1->torchtyping->nvidia-curobo==0.6.2.post1.dev3) (6.8.0)
Requirement already satisfied: attrs>=22.2.0 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (23.1.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2023.7.1)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.3.10)
Requirement already satisfied: referencing>=0.28.4 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (0.30.2)
Requirement already satisfied: rpds-py>=0.7.1 in /home/pren/.local/lib/python3.8/site-packages (from jsonschema->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (0.10.6)
Requirement already satisfied: python-dateutil>=2.2 in /home/pren/.local/lib/python3.8/site-packages (from pycollada->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2.8.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/pren/.local/lib/python3.8/site-packages (from requests->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (3.3.0)
Requirement already satisfied: idna<4,>=2.5 in /home/pren/anaconda3/envs/curobo-env/lib/python3.8/site-packages (from requests->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/pren/.local/lib/python3.8/site-packages (from requests->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /home/pren/anaconda3/envs/curobo-env/lib/python3.8/site-packages (from requests->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (2023.11.17)
Requirement already satisfied: mpmath>=0.19 in /home/pren/.local/lib/python3.8/site-packages (from sympy->trimesh[easy]>=3.11.2->yourdfpy>=0.0.53->nvidia-curobo==0.6.2.post1.dev3) (1.3.0)
Building wheels for collected packages: nvidia-curobo
  Building editable for nvidia-curobo (pyproject.toml) ... done
  Created wheel for nvidia-curobo: filename=nvidia_curobo-0.6.2.post1.dev3-0.editable-cp38-cp38-linux_x86_64.whl size=5627 sha256=0aac1d4d57caa3e70b71eb56820fe595fc2089eac197c2aee23d97614bb6095e
  Stored in directory: /tmp/pip-ephem-wheel-cache-otha34g9/wheels/ac/e5/36/52e2c21d4f654260867554ec1f86a8445280b02c7854b2a741
Successfully built nvidia-curobo
Installing collected packages: lxml, chardet, nvidia-curobo
Successfully installed chardet-5.2.0 lxml-4.9.4 nvidia-curobo-0.6.2.post1.dev3

[notice] A new release of pip is available: 23.3.1 -> 23.3.2
[notice] To update, run: pip install --upgrade pip
(curobo-env) ➜  curobo git:(main) python examples/collision_check_example.py
tensor([[0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.1076],
        [0.1610],
        [0.0000]], device='cuda:0')
Collision Distance:
World: tensor([0.0000, 0.0000, 0.0000, 0.0000, 1.8775], device='cuda:0')
Self: tensor([0., 0., 0., 0., 0.], device='cuda:0')
(curobo-env) ➜  curobo git:(main) python examples/ik_example.py
Success, Solve Time(s), Total Time(s)  0.9986 1.4519603252410889 1453.4814977661533 tensor(0.0187, device='cuda:0') tensor(0.1570, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9946 0.14182186126708984 20139.94180295595 tensor(0.0477, device='cuda:0') tensor(0.5623, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9938 0.14045000076293945 35142.71398719403 tensor(0.0644, device='cuda:0') tensor(0.5433, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9948 0.14042997360229492 35189.83008755709 tensor(0.0345, device='cuda:0') tensor(0.5037, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9952 0.14171576499938965 34891.18300321934 tensor(0.0661, device='cuda:0') tensor(0.4650, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9946 0.14111542701721191 35045.11083445435 tensor(0.0423, device='cuda:0') tensor(0.4917, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9936 0.14092016220092773 35082.16192163377 tensor(0.0604, device='cuda:0') tensor(0.6125, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9948 0.140580415725708 35168.46772516585 tensor(0.0399, device='cuda:0') tensor(0.5153, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.995 0.140869140625 35103.537048537124 tensor(0.0519, device='cuda:0') tensor(0.5363, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9952 0.1402597427368164 35248.266287485334 tensor(0.0485, device='cuda:0') tensor(0.4898, device='cuda:0')
(curobo-env) ➜  curobo git:(main) pip install pluggy
Looking in indexes: https://mirrors.bfsu.edu.cn/pypi/web/simple/
Collecting pluggy
  Using cached https://mirrors.bfsu.edu.cn/pypi/web/packages/05/b8/42ed91898d4784546c5f06c60506400548db3f7a4b3fb441cba4e5c17952/pluggy-1.3.0-py3-none-any.whl (18 kB)
Installing collected packages: pluggy
Successfully installed pluggy-1.3.0

[notice] A new release of pip is available: 23.3.1 -> 23.3.2
[notice] To update, run: pip install --upgrade pip
(curobo-env) ➜  curobo git:(main) python3 -m pytest .
======================================================================================= test session starts =======================================================================================
platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.3.0
rootdir: /home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo, configfile: pyproject.toml
plugins: anyio-4.0.0, torchtyping-0.1.4, typeguard-4.1.5, cov-3.0.0, hydra-core-1.3.2
collected 137 items / 2 skipped                                                                                                                                                                   

tests/cost_test.py .                                                                                                                                                                        [  0%]
tests/cuda_robot_generator_test.py ....                                                                                                                                                     [  3%]
tests/curobo_robot_world_model_test.py ...........                                                                                                                                          [ 11%]
tests/curobo_version_test.py .                                                                                                                                                              [ 12%]
tests/geom_test.py ......                                                                                                                                                                   [ 16%]
tests/geom_types_test.py ......                                                                                                                                                             [ 21%]
tests/goal_test.py ..                                                                                                                                                                       [ 22%]
tests/ik_config_test.py ....                                                                                                                                                                [ 25%]
tests/ik_module_test.py ......                                                                                                                                                              [ 29%]
tests/ik_test.py .....                                                                                                                                                                      [ 33%]
tests/interpolation_test.py .                                                                                                                                                               [ 34%]
tests/kinematics_test.py .....                                                                                                                                                              [ 37%]
tests/motion_gen_api_test.py ..                                                                                                                                                             [ 39%]
tests/motion_gen_module_test.py .............                                                                                                                                               [ 48%]
tests/motion_gen_test.py ...                                                                                                                                                                [ 51%]
tests/mpc_test.py F...                                                                                                                                                                      [ 54%]
tests/multi_pose_test.py .                                                                                                                                                                  [ 54%]
tests/pose_reaching_test.py ....                                                                                                                                                            [ 57%]
tests/pose_test.py ..                                                                                                                                                                       [ 59%]
tests/robot_assets_test.py ...............                                                                                                                                                  [ 70%]
tests/robot_config_test.py ...                                                                                                                                                              [ 72%]
tests/robot_world_model_test.py ............                                                                                                                                                [ 81%]
tests/self_collision_test.py .....                                                                                                                                                          [ 84%]
tests/trajopt_config_test.py ....                                                                                                                                                           [ 87%]
tests/trajopt_test.py ..........                                                                                                                                                            [ 94%]
tests/usd_export_test.py s.                                                                                                                                                                 [ 96%]
tests/warp_mesh_test.py ...                                                                                                                                                                 [ 98%]
tests/world_config_test.py ..                                                                                                                                                               [100%]

============================================================================================ FAILURES =============================================================================================
______________________________________________________________________________ test_mpc_single[mpc_single_env-True] _______________________________________________________________________________

mpc_str = 'mpc_single_env', expected = True, request = <FixtureRequest for <Function test_mpc_single[mpc_single_env-True]>>

    @pytest.mark.parametrize(
        "mpc_str, expected",
        [
            ("mpc_single_env", True),
            # ("mpc_single_env_lbfgs", True), unstable
        ],
    )
    def test_mpc_single(mpc_str, expected, request):
        mpc_val = request.getfixturevalue(mpc_str)
        mpc = mpc_val[0]
        retract_cfg = mpc_val[1]
        start_state = retract_cfg
        state = mpc.rollout_fn.compute_kinematics(JointState.from_position(retract_cfg))
        retract_pose = Pose(state.ee_pos_seq, quaternion=state.ee_quat_seq)
        goal = Goal(
            current_state=JointState.from_position(retract_cfg + 0.5),
            goal_state=JointState.from_position(retract_cfg),
            goal_pose=retract_pose,
        )
        goal_buffer = mpc.setup_solve_single(goal, 1)

        start_state = JointState.from_position(retract_cfg + 0.5, joint_names=mpc.joint_names)

        converged = False
        tstep = 0
        mpc.update_goal(goal_buffer)
        current_state = start_state.clone()
        while not converged:
            result = mpc.step(current_state, max_attempts=1)
            torch.cuda.synchronize()
            current_state.copy_(result.action)
            tstep += 1
            if result.metrics.pose_error.item() < 0.05:
                converged = True
                break
            if tstep > 100:
                break
>       assert converged == expected
E       assert False == True

tests/mpc_test.py:135: AssertionError
======================================================================================== warnings summary =========================================================================================
<unknown>:3
  <unknown>:3: DeprecationWarning: invalid escape sequence \p

tests/ik_config_test.py:149
  /home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/tests/ik_config_test.py:149: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    if expected is not -100:

tests/pose_reaching_test.py::test_pose_sequence_ur5e[True-False-12]
  /home/pren/Space/GitHub-Codes/curobo_ws/curobo/curobo/src/curobo/graph/graph_base.py:696: UserWarning: FALLBACK path has been taken inside: runCudaFusionGroup. This is an indication that codegen Failed for some reason.
  To debug try disable codegen fallback path via setting the env variable `export PYTORCH_NVFUSER_DISABLE=fallback`
   (Triggered internally at  ../torch/csrc/jit/codegen/cuda/manager.cpp:329.)
    return compute_distance_norm_jit(pt, batch_pts, self.distance_weight)

tests/pose_test.py::test_pose_transform_point_grad
  /home/pren/.local/lib/python3.8/site-packages/torch/autograd/gradcheck.py:652: UserWarning: Input #0 requires gradient and is not a double precision floating point or complex. This check will likely fail if all the inputs are not of double precision floating point or complex. 
    warnings.warn(

tests/pose_test.py::test_pose_transform_point_grad
  /home/pren/.local/lib/python3.8/site-packages/torch/autograd/gradcheck.py:652: UserWarning: Input #1 requires gradient and is not a double precision floating point or complex. This check will likely fail if all the inputs are not of double precision floating point or complex. 
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================================================== short test summary info =====================================================================================
FAILED tests/mpc_test.py::test_mpc_single[mpc_single_env-True] - assert False == True
================================================================= 1 failed, 135 passed, 3 skipped, 5 warnings in 74.77s (0:01:14) =================================================================
(curobo-env) ➜  curobo git:(main)

It's working correctly, that one test which failed is an outlier.

It's working correctly, that one test which failed is an outlier.

Thx.

It's working correctly, that one test which failed is an outlier.

By the way, I would like to ask under which environments has CuRobo been tested and is feasible? Have you used CuRobo under Isaac Sim 2023.1.0 (currently 2023.1.0-hotfix.1; and 2023.1.1 is now available) and Ubuntu 22.04? What are the minimum CUDA and pytorch versions required by CuRobo? @balakumar-s

cuRobo will work on pytorch>=1.10 and CUDA>=11.0.

However there might be bugs in pytorch like the one you encountered that can cause issues for newer gpus working on older pytorch.

As long as you can run pytorch with your gpu, curobo will also work.

Regarding Ubuntu, it doesn't matter which one you use.

We have tested isaac sim 2022.2.1 and 2023.1.0. Newer versions should also work.

Hi @balakumar-s , have you encountered these two problems?

python version: 3.10.12
Isaac Sim version (if using): N/A
Ubuntu version: 22.04
GPU: NVIDIA GeForce RTX A6000
Driver version: 525.147.05
CUDA version: 12.1

pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu121

Do I need to use another version of pytorch?

eai@eai-Z790-UD-AX:~/rp/curobo$ nvidia-smi
Mon Dec 25 20:33:36 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.147.05   Driver Version: 525.147.05   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA RTX A6000    Off  | 00000000:01:00.0  On |                  Off |
| 30%   41C    P8    28W / 300W |    365MiB / 49140MiB |     12%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1876      G   /usr/lib/xorg/Xorg                146MiB |
|    0   N/A  N/A      1999      G   /usr/bin/gnome-shell               35MiB |
|    0   N/A  N/A      2480      G   ...RendererForSitePerProcess       13MiB |
|    0   N/A  N/A      2798      G   ...7/usr/lib/firefox/firefox      165MiB |
+-----------------------------------------------------------------------------+
eai@eai-Z790-UD-AX:~/rp/curobo$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Feb__7_19:32:13_PST_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0

Issue 1

eai@eai-Z790-UD-AX:~/rp/curobo$ python3 examples/ik_example.py
Traceback (most recent call last):
  File "/home/eai/rp/curobo/examples/ik_example.py", line 232, in <module>
    demo_basic_ik()
  File "/home/eai/rp/curobo/examples/ik_example.py", line 62, in demo_basic_ik
    result = ik_solver.solve_batch(goal)
  File "/home/eai/rp/curobo/src/curobo/wrap/reacher/ik_solver.py", line 537, in solve_batch
    return self.solve_from_solve_state(
  File "/home/eai/rp/curobo/src/curobo/wrap/reacher/ik_solver.py", line 675, in solve_from_solve_state
    result = self.solver.solve(goal_buffer, coord_position_seed)
  File "/home/eai/rp/curobo/src/curobo/wrap/wrap_base.py", line 139, in solve
    act_seq = self.optimize(seed, shift_steps=0)
  File "/home/eai/rp/curobo/src/curobo/wrap/wrap_base.py", line 71, in optimize
    act_seq = opt.optimize(act_seq, shift_steps)
  File "/home/eai/rp/curobo/src/curobo/opt/opt_base.py", line 96, in optimize
    out = self._optimize(opt_tensor, shift_steps, n_iters)
  File "/home/eai/rp/curobo/src/curobo/opt/newton/newton_base.py", line 144, in _optimize
    best_q, best_cost, q, grad_q = self._call_opt_iters_graph(q, grad_q)
  File "/home/eai/rp/curobo/src/curobo/opt/newton/newton_base.py", line 521, in _call_opt_iters_graph
    return self._opt_iters(q, grad_q)
  File "/home/eai/rp/curobo/src/curobo/opt/newton/newton_base.py", line 170, in _opt_iters
    cost_n, q, grad_q = self._opt_step(q.detach(), grad_q.detach())
  File "/home/eai/rp/curobo/src/curobo/opt/newton/newton_base.py", line 182, in _opt_step
    q_n, cost_n, grad_q_n = self._approx_line_search(q, grad_q)
  File "/home/eai/rp/curobo/src/curobo/opt/newton/newton_base.py", line 425, in _approx_line_search
    return self._wolfe_line_search(x, step_direction)
  File "/home/eai/rp/curobo/src/curobo/opt/newton/newton_base.py", line 333, in _wolfe_line_search
    (best_x_n, best_c_n, best_grad_n) = wolfe_line_search(
  File "/home/eai/rp/curobo/src/curobo/curobolib/ls.py", line 60, in wolfe_line_search
    r = line_search_cu.line_search(
RuntimeError: CUDA error: the provided PTX was compiled with an unsupported toolchain.
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Issue 2

eai@eai-Z790-UD-AX:~/rp/curobo$ python3 -m pytest .
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3/dist-packages/pytest/__main__.py", line 5, in <module>
    raise SystemExit(pytest.console_main())
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/usr/lib/python3/dist-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 92, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 83, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3/dist-packages/_pytest/helpconfig.py", line 100, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 1283, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 1172, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 297, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/usr/lib/python3/dist-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/eai/.local/lib/python3.10/site-packages/typeguard/_pytest_plugin.py", line 6, in <module>
    from pytest import Config, Parser
ImportError: cannot import name 'Config' from 'pytest' (/usr/lib/python3/dist-packages/pytest/__init__.py)

Your driver only supports up to CUDA 12.0. Can you upgrade your nvidia driver and test?

Hi @balakumar-s , thx for your timely reply.

I have updated my CUDA and NVIDIA driver version, and seems that CuRobo works well with Python but seems to have encountered problems in Isaac Sim.

Ubuntu version: 22.04.3 Yython version: 3.10.12 Isaac Sim version (if using): N/A GPU: NVIDIA GeForce RTX A6000 Driver version: 545.29.06 CUDA version: 12.3

eai@eai-Z790-UD-AX:~$ nvidia-smi
Wed Dec 27 19:59:33 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.29.06              Driver Version: 545.29.06    CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA RTX A6000               Off | 00000000:01:00.0  On |                  Off |
| 30%   37C    P8              17W / 300W |    706MiB / 49140MiB |      5%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      2041      G   /usr/lib/xorg/Xorg                          152MiB |
|    0   N/A  N/A      2165      G   /usr/bin/gnome-shell                        225MiB |
|    0   N/A  N/A      2814      G   ...ures=SpareRendererForSitePerProcess       46MiB |
|    0   N/A  N/A     65181      C   python3                                     260MiB |
+---------------------------------------------------------------------------------------+
eai@eai-Z790-UD-AX:~$ 
eai@eai-Z790-UD-AX:~$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:02:13_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0
eai@eai-Z790-UD-AX:~$

sudo apt install git-lfs
git clone https://github.com/NVlabs/curobo.git
cd curobo
git lfs pull
pip install -e . --no-build-isolation

eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$ python3 examples/ik_example.py
/usr/lib/python3/dist-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.26.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
Success, Solve Time(s), Total Time(s)  0.9992 1.1695911884307861 1789.24586747862 tensor(0.0107, device='cuda:0') tensor(0.1423, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9972 0.2578887939453125 16037.759801289963 tensor(0.0233, device='cuda:0') tensor(0.3517, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9942 0.25640439987182617 19382.611074552486 tensor(0.0702, device='cuda:0') tensor(0.5208, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9946 0.2590043544769287 19212.834745867523 tensor(0.0736, device='cuda:0') tensor(0.4925, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9944 0.25742006301879883 19333.383115260833 tensor(0.0466, device='cuda:0') tensor(0.5087, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9938 0.258974552154541 19223.24294694696 tensor(0.0391, device='cuda:0') tensor(0.5017, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.993 0.25754809379577637 19317.9427375511 tensor(0.0525, device='cuda:0') tensor(0.6029, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.996 0.2578907012939453 19299.27630658686 tensor(0.0280, device='cuda:0') tensor(0.3821, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9948 0.2580077648162842 19290.435691250303 tensor(0.0484, device='cuda:0') tensor(0.4698, device='cuda:0')
Success, Solve Time(s), Total Time(s)  0.9942 0.25784993171691895 19304.072799706915 tensor(0.0520, device='cuda:0') tensor(0.5610, device='cuda:0')
eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$

eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$ python3 examples/collision_check_example.py
/usr/lib/python3/dist-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.26.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
tensor([[0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.0000],
        [0.0922],
        [0.0000],
        [0.0000],
        [0.0000]], device='cuda:0')
Collision Distance:
World: tensor([0.0000, 0.0000, 0.0000, 0.0000, 1.8775], device='cuda:0')
Self: tensor([0., 0., 0., 0., 0.], device='cuda:0')
eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$

eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$ python3 examples/kinematics_example.py 
/usr/lib/python3/dist-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.26.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
[INFO] [curobo] Self Collision threads, skipped %: 60.67307692307692
[INFO] [curobo] Self Collision count: 818.0
[INFO] [curobo] Self Collision per thread: 0.798828125
[INFO] [curobo] Self Collision using: 2
eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$

eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$ python3 examples/motion_gen_example.py 
/usr/lib/python3/dist-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.26.2
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
Trajectory Generated:  tensor([True, True], device='cuda:0')
eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$

Pytest Error

eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$ python3 -m pytest .
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3/dist-packages/pytest/__main__.py", line 5, in <module>
    raise SystemExit(pytest.console_main())
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/usr/lib/python3/dist-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 92, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 83, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3/dist-packages/_pytest/helpconfig.py", line 100, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3/dist-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 1283, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 1172, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/usr/lib/python3/dist-packages/pluggy/manager.py", line 297, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/usr/lib/python3/dist-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/eai/.local/lib/python3.10/site-packages/typeguard/_pytest_plugin.py", line 6, in <module>
    from pytest import Config, Parser
ImportError: cannot import name 'Config' from 'pytest' (/usr/lib/python3/dist-packages/pytest/__init__.py)
eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$

Isaac Sim Error

I first updated the version of torch in Isaac Sim, then CuRobo is compiled normally. But among all the Isaac Sim examples, only omni_python examples/isaac_sim/collision_checker.py can be executed well, and other examples have the same error.

eai@eai-Z790-UD-AX:~/rp/build_curobo/for_isaac_sim/curobo$ omni_python -m pip list | grep torch
torch                                2.1.0+cu121
torchaudio                           2.1.0+cu121
torchtyping                          0.1.4
torchvision                          0.16.0+cu121
WARNING: You are using pip version 21.2.1+nv1; however, version 23.3.2 is available.
You should consider upgrading via the '/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/bin/python3 -m pip install --upgrade pip' command.

omni_python examples/isaac_sim/multi_arm_reacher.py --robot dual_ur10e.yml  # Rrror

[5.967s] Simulation App Starting
2023-12-27 11:20:59 [5,972ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/ecoMode/enabled has been requested but is not available
2023-12-27 11:20:59 [5,972ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/directLighting/enabled has been requested but is not available
2023-12-27 11:20:59 [5,973ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/reflections/enabled has been requested but is not available
2023-12-27 11:20:59 [5,973ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/translucency/enabled has been requested but is not available
2023-12-27 11:20:59 [5,973ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/raytracing/subsurface/enabled has been requested but is not available
2023-12-27 11:20:59 [5,973ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/caustics/enabled has been requested but is not available
2023-12-27 11:21:00 [6,969ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-27 11:21:00 [6,969ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[7.637s] app ready
2023-12-27 11:21:01 [7,995ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-27 11:21:01 [7,995ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[8.190s] RTX ready
[8.330s] Simulation App Startup Complete
2023-12-27 11:21:02 [9,154ms] [Warning] [omni.importer.urdf] The path wrist_3-flange is not a valid usd path, modifying to wrist_3_flange
2023-12-27 11:21:02 [9,154ms] [Warning] [omni.importer.urdf] The path flange-tool0 is not a valid usd path, modifying to flange_tool0
2023-12-27 11:21:02 [9,154ms] [Warning] [omni.importer.urdf] The path wrist_3-flange is not a valid usd path, modifying to wrist_3_flange
2023-12-27 11:21:02 [9,154ms] [Warning] [omni.importer.urdf] The path flange-tool0 is not a valid usd path, modifying to flange_tool0
2023-12-27 11:21:02 [9,156ms] [Warning] [omni.importer.urdf] Link base_fixture_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:21:02 [9,157ms] [Warning] [omni.importer.urdf] Link base_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:21:02 [9,236ms] [Warning] [omni.importer.urdf] Link flange has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:21:02 [9,238ms] [Warning] [omni.importer.urdf] Link tool0 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:21:02 [9,239ms] [Warning] [omni.importer.urdf] Link base_link_1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:21:02 [9,314ms] [Warning] [omni.importer.urdf] Link flange_1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:21:02 [9,315ms] [Warning] [omni.importer.urdf] Link tool1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
Traceback (most recent call last):
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 354, in <module>
    main()
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 118, in main
    robot, robot_prim_path = add_robot_to_scene(robot_cfg, my_world)
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/helper.py", line 115, in add_robot_to_scene
    robot_p.set_solver_velocity_iteration_count(4)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation.py", line 981, in set_solver_velocity_iteration_count
    self._articulation_view.set_solver_velocity_iteration_counts(count)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation_view.py", line 3243, in set_solver_velocity_iteration_counts
    set_prim_property(self.prim_paths[i], "physxArticulation:solverVelocityIterationCount", counts[read_idx])
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/prims.py", line 802, in set_prim_property
    prim.GetAttribute(property_name).Set(property_value)
pxr.Tf.ErrorException: 
    Error in 'pxrInternal_v0_22__pxrReserved__::UsdStage::_SetValueImpl' at line 6174 in file /buildAgent/work/ac88d7d902b57417/USD/pxr/usd/usd/stage.cpp : 'Empty typeName for </World/ur10e_robot.physxArticulation:solverVelocityIterationCount>'
2023-12-27 11:21:02 [9,329ms] [Warning] [carb] [Plugin: omni.spectree.delegate.plugin] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.usd_resolver/bin/libomni.spectree.delegate.plugin.so remained loaded after unload request
2023-12-27 11:21:02 [9,331ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:21:02 [9,332ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal
2023-12-27 11:21:02 [9,335ms] [Warning] [carb.audio.context] 1 contexts were leaked
2023-12-27 11:21:02 [9,385ms] [Warning] [carb] Recursive unloadAllPlugins() detected!
2023-12-27 11:21:02 [9,398ms] [Warning] [omni.core.ITypeFactory] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.activity.core/bin/libomni.activity.core.plugin.so remained loaded after unload request.
eai@eai-Z790-UD-AX:~/rp/build_curobo/for_isaac_sim/curobo$

omni_python examples/isaac_sim/mpc_example.py --robot franka.yml  # Error

[5.924s] Simulation App Starting
2023-12-27 11:21:55 [5,930ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/ecoMode/enabled has been requested but is not available
2023-12-27 11:21:55 [5,930ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/directLighting/enabled has been requested but is not available
2023-12-27 11:21:55 [5,930ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/reflections/enabled has been requested but is not available
2023-12-27 11:21:55 [5,930ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/translucency/enabled has been requested but is not available
2023-12-27 11:21:55 [5,930ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/raytracing/subsurface/enabled has been requested but is not available
2023-12-27 11:21:55 [5,930ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/caustics/enabled has been requested but is not available
2023-12-27 11:21:56 [6,934ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-27 11:21:56 [6,935ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[7.605s] app ready
2023-12-27 11:21:57 [7,953ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-27 11:21:57 [7,953ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[8.082s] RTX ready
[8.312s] Simulation App Startup Complete
2023-12-27 11:22:18 [29,152ms] [Warning] [omni.client.python] Detected a blocking function. This will cause hitches or hangs in the UI. Please switch to the async version:
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/mpc_example.py", line 376, in <module>
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/mpc_example.py", line 148, in main
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/scenes/scene.py", line 225, in add_default_ground_plane
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/nucleus.py", line 572, in get_assets_root_path
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/nucleus.py", line 192, in check_server
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/extscore/omni.client/omni/client/__init__.py", line 158, in stat

2023-12-27 11:24:11 [142,006ms] [Warning] [omni.importer.urdf] Link base_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:24:11 [142,157ms] [Warning] [omni.importer.urdf] Link ee_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:24:11 [142,172ms] [Warning] [omni.importer.urdf] Link right_gripper has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
Traceback (most recent call last):
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/mpc_example.py", line 376, in <module>
    main()
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/mpc_example.py", line 178, in main
    robot, robot_prim_path = add_robot_to_scene(robot_cfg, my_world)
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/helper.py", line 115, in add_robot_to_scene
    robot_p.set_solver_velocity_iteration_count(4)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation.py", line 981, in set_solver_velocity_iteration_count
    self._articulation_view.set_solver_velocity_iteration_counts(count)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation_view.py", line 3243, in set_solver_velocity_iteration_counts
    set_prim_property(self.prim_paths[i], "physxArticulation:solverVelocityIterationCount", counts[read_idx])
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/prims.py", line 802, in set_prim_property
    prim.GetAttribute(property_name).Set(property_value)
pxr.Tf.ErrorException: 
    Error in 'pxrInternal_v0_22__pxrReserved__::UsdStage::_SetValueImpl' at line 6174 in file /buildAgent/work/ac88d7d902b57417/USD/pxr/usd/usd/stage.cpp : 'Empty typeName for </World/panda.physxArticulation:solverVelocityIterationCount>'
2023-12-27 11:24:11 [142,183ms] [Warning] [carb] [Plugin: omni.spectree.delegate.plugin] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.usd_resolver/bin/libomni.spectree.delegate.plugin.so remained loaded after unload request
2023-12-27 11:24:11 [142,185ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:24:11 [142,186ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal
2023-12-27 11:24:11 [142,188ms] [Warning] [carb.audio.context] 1 contexts were leaked
2023-12-27 11:24:11 [142,226ms] [Warning] [carb] Recursive unloadAllPlugins() detected!
2023-12-27 11:24:11 [142,237ms] [Warning] [omni.core.ITypeFactory] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.activity.core/bin/libomni.activity.core.plugin.so remained loaded after unload request.

omni_python examples/isaac_sim/ik_reachability.py --robot franka.yml  # Error

[5.806s] Simulation App Starting
2023-12-27 11:26:50 [5,815ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/ecoMode/enabled has been requested but is not available
2023-12-27 11:26:50 [5,815ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/directLighting/enabled has been requested but is not available
2023-12-27 11:26:50 [5,815ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/reflections/enabled has been requested but is not available
2023-12-27 11:26:50 [5,815ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/translucency/enabled has been requested but is not available
2023-12-27 11:26:50 [5,815ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/raytracing/subsurface/enabled has been requested but is not available
2023-12-27 11:26:50 [5,815ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/caustics/enabled has been requested but is not available
2023-12-27 11:26:51 [6,796ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-27 11:26:51 [6,796ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[7.479s] app ready
2023-12-27 11:26:52 [7,833ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-27 11:26:52 [7,833ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[8.023s] RTX ready
[8.170s] Simulation App Startup Complete
2023-12-27 11:26:53 [9,016ms] [Warning] [omni.importer.urdf] Link base_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:26:53 [9,171ms] [Warning] [omni.importer.urdf] Link ee_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-27 11:26:53 [9,187ms] [Warning] [omni.importer.urdf] Link right_gripper has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
Traceback (most recent call last):
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/ik_reachability.py", line 366, in <module>
    main()
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/ik_reachability.py", line 168, in main
    robot, robot_prim_path = add_robot_to_scene(robot_cfg, my_world)
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/helper.py", line 115, in add_robot_to_scene
    robot_p.set_solver_velocity_iteration_count(4)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation.py", line 981, in set_solver_velocity_iteration_count
    self._articulation_view.set_solver_velocity_iteration_counts(count)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation_view.py", line 3243, in set_solver_velocity_iteration_counts
    set_prim_property(self.prim_paths[i], "physxArticulation:solverVelocityIterationCount", counts[read_idx])
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/prims.py", line 802, in set_prim_property
    prim.GetAttribute(property_name).Set(property_value)
pxr.Tf.ErrorException: 
    Error in 'pxrInternal_v0_22__pxrReserved__::UsdStage::_SetValueImpl' at line 6174 in file /buildAgent/work/ac88d7d902b57417/USD/pxr/usd/usd/stage.cpp : 'Empty typeName for </World/panda.physxArticulation:solverVelocityIterationCount>'
2023-12-27 11:26:53 [9,198ms] [Warning] [carb] [Plugin: omni.spectree.delegate.plugin] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.usd_resolver/bin/libomni.spectree.delegate.plugin.so remained loaded after unload request
2023-12-27 11:26:53 [9,199ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-27 11:26:53 [9,200ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal
2023-12-27 11:26:53 [9,202ms] [Warning] [carb.audio.context] 1 contexts were leaked
2023-12-27 11:26:53 [9,233ms] [Warning] [carb] Recursive unloadAllPlugins() detected!
2023-12-27 11:26:53 [9,239ms] [Warning] [omni.core.ITypeFactory] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.activity.core/bin/libomni.activity.core.plugin.so remained loaded after unload request.
eai@eai-Z790-UD-AX:~/rp/build_curobo/for_isaac_sim/curobo$

I am not sure what's happening here. Did you install usd-core in the isaac sim python environment (this can break things as Isaac Sim comes with it's own usd-core) ?

Try one of the below:

Option 1: We have not tested in Isaac Sim 2023.1.1.

Can you comment out this line https://github.com/NVlabs/curobo/blob/c09d94908d8ad52f557ed59195c1ceb2d0434d65/examples/isaac_sim/helper.py#L115 and also the next line and see if the errors go away?

Option 2: We do not install a newer pytorch in an Isaac Sim env. Isaac sim comes with pytorch that was compiled with cuda 11.8. If you install cuda 11.8 in your machine that might fix the problem. Though I don't think this is the cause of the problem.

Regarding your pytest error, we have a known issue with pytest == 6.2.5 (https://curobo.org/source/getting_started/6_known_issues.html#python). Installing any other version of pytest will work.

Regarding your pytest error, we have a known issue with pytest == 6.2.5 (https://curobo.org/source/getting_started/6_known_issues.html#python). Installing any other version of pytest will work.

Hi @balakumar-s , this solution works. The following output proves that pytest can be passed, right?

eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$ pip list | grep pytest
launch-pytest                        1.0.4
pytest                               6.2.5
pytest-cov                           3.0.0
pytest-mock                          3.6.1
pytest-timeout                       2.1.0
eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$ pip install pytest==7.1.1
Defaulting to user installation because normal site-packages is not writeable
Collecting pytest==7.1.1
  Downloading pytest-7.1.1-py3-none-any.whl (297 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 297.0/297.0 kB 1.2 MB/s eta 0:00:00
Requirement already satisfied: attrs>=19.2.0 in /home/eai/.local/lib/python3.10/site-packages (from pytest==7.1.1) (23.1.0)
Requirement already satisfied: iniconfig in /usr/lib/python3/dist-packages (from pytest==7.1.1) (1.1.1)
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from pytest==7.1.1) (21.3)
Requirement already satisfied: pluggy<2.0,>=0.12 in /usr/lib/python3/dist-packages (from pytest==7.1.1) (0.13.0)
Requirement already satisfied: py>=1.8.2 in /usr/lib/python3/dist-packages (from pytest==7.1.1) (1.10.0)
Requirement already satisfied: tomli>=1.0.0 in /usr/lib/python3/dist-packages (from pytest==7.1.1) (1.2.2)
Installing collected packages: pytest
Successfully installed pytest-7.1.1
eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$ pip list | grep pytest
launch-pytest                        1.0.4
pytest                               7.1.1
pytest-cov                           3.0.0
pytest-mock                          3.6.1
pytest-timeout                       2.1.0
eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$ python3 -m pytest .
=========================================================================== test session starts ===========================================================================
platform linux -- Python 3.10.12, pytest-7.1.1, pluggy-0.13.0
rootdir: /home/eai/rp/build_curobo/curobo, configfile: pyproject.toml
plugins: ament-mypy-0.12.9, ament-lint-0.12.9, launch-pytest-1.0.4, ament-flake8-0.12.9, ament-copyright-0.12.9, launch-testing-1.0.4, ament-pep257-0.12.9, launch-testing-ros-0.19.6, ament-xmllint-0.12.9, torchtyping-0.1.4, anyio-4.2.0, typeguard-4.1.5, mock-3.6.1, timeout-2.1.0, cov-3.0.0, colcon-core-0.15.1
collected 0 items / 1 skipped                                                                                                                                             

============================================================================ warnings summary =============================================================================
../../../../../usr/lib/python3/dist-packages/scipy/__init__.py:146
  Warning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.26.2

<unknown>:3
  Warning: invalid escape sequence '\p'

tests/ik_config_test.py:149
  Warning: "is not" with a literal. Did you mean "!="?

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================================== 1 skipped, 3 warnings in 1.57s ======================================================================
eai@eai-Z790-UD-AX:~/rp/build_curobo/curobo$

No, it didn't find any tests. Can you try with python3 -m pytest tests/ ?

I am not sure what's happening here. Did you install usd-core in the isaac sim python environment (this can break things as Isaac Sim comes with it's own usd-core) ?

Try one of the below:

Option 1: We have not tested in Isaac Sim 2023.1.1.

Can you comment out this line

https://github.com/NVlabs/curobo/blob/c09d94908d8ad52f557ed59195c1ceb2d0434d65/examples/isaac_sim/helper.py#L115

and also the next line and see if the errors go away?

Option 2: We do not install a newer pytorch in an Isaac Sim env. Isaac sim comes with pytorch that was compiled with cuda 11.8. If you install cuda 11.8 in your machine that might fix the problem. Though I don't think this is the cause of the problem.

Hi @balakumar-s , I just install usd-core in the Isaac Sim Python environment, but it does not work.

eai@eai-Z790-UD-AX:~/rp/build_curobo/for_isaac_sim/curobo$ omni_python -m pip install usd-core
Collecting usd-core
  Downloading usd_core-23.11-cp310-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25.8 MB)
     |████████████████████████████████| 25.8 MB 600 kB/s 
Installing collected packages: usd-core
Successfully installed usd-core-23.11
WARNING: You are using pip version 21.2.1+nv1; however, version 23.3.2 is available.
You should consider upgrading via the '/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/bin/python3 -m pip install --upgrade pip' command.
eai@eai-Z790-UD-AX:~/rp/build_curobo/for_isaac_sim/curobo$ omni_python examples/isaac_sim/multi_arm_reacher.py --robot dual_ur10e.yml
Starting kit application with the following args:  ['/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.kit/omni/isaac/kit/simulation_app.py', '/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/apps/omni.isaac.sim.python.kit', '--/app/tokens/exe-path=/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit', '--/persistent/app/viewport/displayOptions=3094', '--/rtx/materialDb/syncLoads=True', '--/rtx/hydra/materialSyncLoads=True', '--/omni.kit.plugin/syncUsdLoads=True', '--/app/renderer/resolution/width=1920', '--/app/renderer/resolution/height=1080', '--/app/window/width=1440', '--/app/window/height=900', '--/renderer/multiGpu/enabled=True', '--/app/fastShutdown=True', '--ext-folder', '/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts', '--ext-folder', '/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/apps', '--/physics/cudaDevice=0', '--portable']
Passing the following args to the base kit application:  ['--robot', 'dual_ur10e.yml']
[Info] [carb] Logging to file: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/logs/Kit/Isaac-Sim/2023.1/kit_20231228_090404.log
2023-12-28 01:04:04 [0ms] [Warning] [omni.kit.app.plugin] No crash reporter present, dumps uploading isn't available.
[0.027s] [ext: omni.kit.async_engine-0.0.0] startup
[0.160s] [ext: omni.activity.core-1.0.1] startup
[0.162s] [ext: omni.assets.plugins-0.0.0] startup
[0.163s] [ext: omni.stats-0.0.0] startup
[0.163s] [ext: omni.client-1.0.1] startup
[0.167s] [ext: omni.activity.profiler-1.0.2] startup
[0.168s] [ext: omni.gpu_foundation-0.0.0] startup
[0.173s] [ext: omni.rtx.shadercache.vulkan-1.0.0] startup
[0.174s] [ext: carb.windowing.plugins-1.0.0] startup
[0.178s] [ext: omni.kit.renderer.init-0.0.0] startup
2023-12-28 01:04:04 [433ms] [Warning] [gpu.foundation.plugin] Skipping unsupported non-NVIDIA GPU: Intel(R) Graphics (RPL-S)
2023-12-28 01:04:04 [433ms] [Warning] [gpu.foundation.plugin] Skipping unsupported non-NVIDIA GPU: Intel(R) Graphics (RPL-S)

|---------------------------------------------------------------------------------------------|
| Driver Version: 545.29.06     | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|     |                                  |        |     |            | Bus-ID    |            |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA RTX A6000                 | Yes: 0 |     | 49386   MB | 10de      | 0          |
|     |                                  |        |     |            | 2230      | 14a8ce36.. |
|     |                                  |        |     |            | 1         |            |
|---------------------------------------------------------------------------------------------|
| 1   | Intel(R) Graphics (RPL-S)        |        |     | 96430   MB | 8086      | 0          |
|     |                                  |        |     |            | a780      | 21a1fe4f.. |
|     |                                  |        |     |            | 0         |            |
|=============================================================================================|
| OS: 22.04.3 LTS (Jammy Jellyfish) ubuntu, Version: 22.04.3, Kernel: 6.2.0-39-generic
| XServer Vendor: The X.Org Foundation, XServer Version: 12101004 (1.21.1.4)
| Processor: 13th Gen Intel(R) Core(TM) i9-13900K | Cores: 24 | Logical: 48
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 128573 | Free Memory: 121345
| Total Page/Swap (MB): 2047 | Free Page/Swap: 2047
|---------------------------------------------------------------------------------------------|
[0.686s] [ext: omni.kit.pipapi-0.0.0] startup
[0.687s] [ext: omni.kit.pip_archive-0.0.0] startup
[0.688s] [ext: omni.pip.compute-1.2.0] startup
[0.688s] [ext: omni.pip.cloud-1.0.1] startup
[0.688s] [ext: omni.isaac.core_archive-2.2.1] startup
[0.689s] [ext: omni.pip.torch-2_0_1-2.0.2] startup
[0.706s] [ext: omni.kit.telemetry-0.5.0] startup
[0.724s] [ext: omni.kit.loop-isaac-1.1.0] startup
[0.725s] [ext: omni.kit.test-0.0.0] startup
[0.726s] [ext: omni.isaac.ml_archive-1.1.3] startup
[0.726s] [ext: omni.mtlx-0.1.0] startup
[0.727s] [ext: omni.usd.config-1.0.3] startup
[0.730s] [ext: omni.gpucompute.plugins-0.0.0] startup
[0.730s] [ext: omni.usd.libs-1.0.0] startup
[0.786s] [ext: omni.appwindow-1.1.5] startup
[1.180s] [ext: omni.kit.renderer.core-0.0.0] startup
[3.918s] [ext: omni.kit.renderer.capture-0.0.0] startup
[3.919s] [ext: omni.kit.renderer.imgui-0.0.0] startup
[4.027s] [ext: carb.audio-0.1.0] startup
[4.028s] [ext: omni.ui-2.18.5] startup
[4.034s] [ext: omni.uiaudio-1.0.0] startup
[4.035s] [ext: omni.kit.mainwindow-1.0.1] startup
[4.036s] [ext: omni.kit.uiapp-0.0.0] startup
[4.036s] [ext: omni.usd.schema.physics-0.0.0] startup
[4.101s] [ext: omni.usd.schema.audio-0.0.0] startup
[4.105s] [ext: omni.usd.schema.forcefield-0.0.0] startup
[4.109s] [ext: omni.usd.schema.omnigraph-1.0.0] startup
[4.113s] [ext: omni.usd.schema.scene.visualization-2.0.2] startup
[4.114s] [ext: omni.usd.schema.semantics-0.0.0] startup
[4.119s] [ext: omni.usd.schema.physx-0.0.0] startup
[4.134s] [ext: omni.usd.schema.anim-0.0.0] startup
[4.144s] [ext: omni.anim.navigation.schema-105.2.0] startup
[4.146s] [ext: omni.anim.graph.schema-105.2.2] startup
[4.150s] [ext: omni.usd.schema.isaac-2.0.1] startup
[4.154s] [ext: omni.usd.schema.geospatial-0.0.0] startup
[4.156s] [ext: omni.usd.schema.omniscripting-1.0.0] startup
[4.159s] [ext: omni.resourcemonitor-105.0.0] startup
[4.160s] [ext: omni.timeline-1.0.9] startup
[4.161s] [ext: omni.usd_resolver-1.0.1] startup
[4.164s] [ext: omni.kit.window.popup_dialog-2.0.23] startup
[4.166s] [ext: omni.graph.exec-0.3.0] startup
[4.167s] [ext: omni.kit.actions.core-1.0.0] startup
[4.167s] [ext: omni.usd.core-1.1.8] startup
[4.170s] [ext: omni.kit.widget.nucleus_connector-1.1.4] startup
[4.171s] [ext: omni.kit.audiodeviceenum-1.0.1] startup
[4.172s] [ext: omni.kit.exec.core-0.5.0] startup
[4.173s] [ext: omni.kit.commands-1.4.6] startup
[4.175s] [ext: omni.hydra.usdrt_delegate-7.2.34] startup
[4.185s] [ext: omni.hydra.scene_delegate-0.3.2] startup
[4.190s] [ext: omni.usd-1.10.18] startup
[4.221s] [ext: omni.kit.widget.versioning-1.4.6] startup
[4.222s] [ext: omni.kit.helper.file_utils-0.1.6] startup
[4.233s] [ext: omni.kit.widget.nucleus_info-1.0.2] startup
[4.233s] [ext: omni.kit.widget.filebrowser-2.3.35] startup
[4.235s] [ext: omni.kit.search_core-1.0.5] startup
[4.235s] [ext: omni.kit.widget.path_field-2.0.8] startup
[4.236s] [ext: omni.kit.widget.search_delegate-1.0.4] startup
[4.236s] [ext: omni.kit.widget.browser_bar-2.0.9] startup
[4.236s] [ext: omni.kit.collaboration.telemetry-1.0.0] startup
[4.237s] [ext: omni.kit.menu.utils-1.5.6] startup
[4.242s] [ext: omni.kit.collaboration.channel_manager-1.0.11] startup
[4.242s] [ext: omni.ui.scene-1.7.0] startup
[4.245s] [ext: omni.kit.notification_manager-1.0.6] startup
[4.245s] [ext: omni.kit.clipboard-1.0.3] startup
[4.246s] [ext: omni.kit.widget.options_menu-1.0.5] startup
[4.246s] [ext: omni.kit.usd.layers-2.1.27] startup
[4.252s] [ext: omni.kit.window.filepicker-2.10.13] startup
[4.256s] [ext: omni.kit.widget.filter-1.1.2] startup
[4.256s] [ext: omni.kit.menu.create-1.0.11] startup
[4.257s] [ext: omni.kit.window.file_exporter-1.0.22] startup
[4.258s] [ext: omni.kit.hotkeys.core-1.3.0] startup
[4.260s] [ext: omni.kit.context_menu-1.6.8] startup
[4.261s] [ext: omni.kit.widget.stage-2.9.6] startup
[4.265s] [ext: omni.kit.primitive.mesh-1.0.15] startup
[4.267s] [ext: omni.kit.usd_undo-0.1.2] startup
[4.267s] [ext: omni.kit.stage_templates-1.1.20] startup
[4.268s] [ext: omni.kit.widget.searchfield-1.1.3] startup
[4.268s] [ext: omni.kit.widget.highlight_label-1.0.1] startup
[4.268s] [ext: omni.kit.window.file_importer-1.0.22] startup
[4.269s] [ext: omni.kit.window.property-1.9.6] startup
[4.270s] [ext: omni.ui_query-1.1.1] startup
[4.270s] [ext: omni.inspect-1.0.1] startup
[4.271s] [ext: usdrt.scenegraph-7.2.34] startup
[4.295s] [ext: omni.graph.tools-1.41.3] startup
[4.303s] [ext: omni.graph.core-2.139.12] startup
[4.306s] [ext: omni.kit.window.file-1.3.44] startup
[4.307s] [ext: omni.graph-1.106.1] startup
[4.329s] [ext: omni.kvdb-0.0.0] startup
[4.330s] [ext: omni.convexdecomposition-105.1.12-5.1] startup
[4.333s] [ext: omni.localcache-0.0.0] startup
[4.334s] [ext: omni.iray.libs-0.0.0] startup
[4.337s] [ext: omni.physx.foundation-105.1.12-5.1] startup
[4.337s] [ext: omni.usdphysics-105.1.12-5.1] startup
[4.340s] [ext: omni.mdl.neuraylib-0.2.0] startup
[4.341s] [ext: omni.isaac.version-1.0.3] startup
[4.341s] [ext: omni.physx.cooking-105.1.12-5.1] startup
[4.347s] [ext: omni.mdl-0.2.1] startup
[4.357s] [ext: omni.physics.tensors-0.1.0] startup
[4.362s] [ext: omni.physx-105.1.12-5.1] startup
2023-12-28 01:04:08 [4,361ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
[4.371s] [ext: omni.kit.material.library-1.3.41] startup
[4.374s] [ext: omni.volume-0.4.0] startup
[4.376s] [ext: omni.physx.tensors-0.1.0] startup
[4.379s] [ext: omni.kit.numpy.common-0.1.2] startup
[4.380s] [ext: omni.hydra.rtx-0.1.0] startup
[4.387s] [ext: omni.warp.core-1.0.0-beta.2] startup
Warp 1.0.0-beta.2 initialized:
   CUDA Toolkit: 11.5, Driver: 12.3
   Devices:
     "cpu"    | x86_64
     "cuda:0" | NVIDIA RTX A6000 (sm_86)
   Kernel cache: /home/eai/.cache/warp/1.0.0-beta.2
[4.462s] [ext: omni.isaac.dynamic_control-1.2.6] startup
[4.473s] [ext: omni.debugdraw-0.1.2] startup
[4.477s] [ext: omni.isaac.core-3.3.2] startup
[4.633s] [ext: omni.hydra.pxr-1.1.6] startup
[4.637s] [ext: omni.kit.collaboration.presence_layer-1.0.8] startup
[4.638s] [ext: omni.kit.widget.prompt-1.0.6] startup
[4.639s] [ext: omni.kit.hydra_texture-1.1.12] startup
[4.643s] [ext: omni.hydra.engine.stats-1.0.1] startup
[4.647s] [ext: omni.kit.viewport.legacy_gizmos-1.0.14] startup
[4.650s] [ext: omni.kit.widget.viewport-105.1.7] startup
[4.653s] [ext: omni.kit.widget.searchable_combobox-1.0.5] startup
[4.654s] [ext: omni.kit.window.drop_support-1.0.1] startup
[4.654s] [ext: omni.kit.viewport.registry-104.0.5] startup
[4.655s] [ext: omni.kit.widget.settings-1.0.6] startup
[4.656s] [ext: omni.kit.window.content_browser_registry-0.0.1] startup
[4.657s] [ext: omni.kit.viewport.window-105.1.12] startup
[4.668s] [ext: omni.kit.window.preferences-1.3.23] startup
[4.672s] [ext: omni.kit.widget.graph-1.8.4] startup
[4.674s] [ext: omni.kit.window.content_browser-2.7.17] startup
[4.681s] [ext: omni.kit.viewport.utility-1.0.16] startup
[4.682s] [ext: omni.kit.widget.live_session_management-1.2.22] startup
[4.684s] [ext: omni.kit.window.extensions-1.3.0] startup
[4.687s] [ext: omni.kit.widget.text_editor-1.0.2] startup
[4.688s] [ext: omni.kit.property.usd-3.21.15] startup
[4.694s] [ext: omni.kit.manipulator.viewport-104.0.9] startup
[4.695s] [ext: omni.graph.ui-1.48.0] startup
[4.705s] [ext: omni.kit.ui_test-1.2.13] startup
[4.706s] [ext: omni.graph.visualization.nodes-2.1.1] startup
[4.714s] [ext: omni.graph.action-1.55.1] startup
[4.738s] [ext: omni.isaac.ui-0.13.5] startup
[4.743s] [ext: omni.isaac.surface_gripper-0.7.0] startup
[4.750s] [ext: omni.isaac.lula-2.1.0] startup
[4.756s] [ext: omni.kit.graph.delegate.default-1.1.1] startup
[4.757s] [ext: omni.isaac.manipulators-1.1.0] startup
[4.761s] [ext: omni.isaac.motion_generation-6.1.0] startup
[4.765s] [ext: omni.kit.graph.editor.core-1.5.1] startup
[4.766s] [ext: omni.kit.graph.delegate.modern-1.9.1] startup
[4.767s] [ext: omni.isaac.franka-0.4.0] startup
[4.769s] [ext: omni.kit.graph.widget.variables-2.1.0] startup
[4.770s] [ext: omni.kit.graph.usd.commands-1.3.0] startup
[4.771s] [ext: omni.isaac.universal_robots-0.3.3] startup
[4.773s] [ext: omni.isaac.cortex-0.3.5] startup
[4.775s] [ext: omni.graph.window.core-1.84.0] startup
[4.781s] [ext: omni.isaac.dofbot-0.3.1] startup
[4.783s] [ext: omni.isaac.cortex.sample_behaviors-1.0.4] startup
[4.785s] [ext: omni.graph.window.generic-1.9.3] startup
[4.788s] [ext: omni.isaac.kit-1.4.7] startup
[4.788s] [ext: omni.graph.window.action-1.11.2] startup
[4.792s] [ext: omni.kit.widget.zoombar-1.0.5] startup
[4.793s] [ext: omni.graph.image.core-0.1.0] startup
[4.793s] [ext: omni.kit.window.cursor-1.1.2] startup
[4.794s] [ext: omni.kit.browser.core-2.3.10] startup
[4.796s] [ext: omni.kit.widget.spinner-1.0.6] startup
[4.798s] [ext: omni.graph.nodes-1.106.3] startup
[5.064s] [ext: omni.kit.widget.material_preview-1.0.8] startup
[5.071s] [ext: omni.kit.browser.folder.core-1.9.7] startup
[5.080s] [ext: omni.kit.viewport.menubar.core-105.0.19] startup
[5.093s] [ext: omni.kit.renderer.cuda_interop-1.0.1] startup
[5.094s] [ext: omni.kit.window.material_graph-1.8.11] startup
[5.110s] [ext: omni.kit.browser.sample-1.4.6] startup
[5.112s] [ext: omni.syntheticdata-0.4.10] startup
[5.128s] [ext: omni.kit.livestream.core-2.3.0] startup
[5.129s] [ext: omni.kit.streamsdk.plugins-2.5.2] startup
[5.130s] [ext: omni.warp-1.0.0-beta.2] startup
[5.171s] [ext: omni.graph.scriptnode-1.1.6] startup
[5.175s] [ext: omni.command.usd-1.0.2] startup
[5.176s] [ext: omni.kit.livestream.native-2.4.0] startup

Active user not found. Using default user [kiosk]Streaming server started.
[5.244s] [ext: omni.replicator.core-1.10.20] startup
2023-12-28 01:04:09 [5,250ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
[5.360s] [ext: omni.kit.viewport.menubar.lighting-105.0.7] startup
[5.362s] [ext: omni.isaac.urdf-1.1.0] startup
2023-12-28 01:04:09 [5,353ms] [Warning] [omni.isaac.urdf] The extension omni.isaac.urdf has been deprecated. Please use omni.importer.urdf instead.
[5.362s] [ext: omni.isaac.core_nodes-1.7.1] startup
[5.380s] [ext: omni.isaac.cloner-0.7.2] startup
[5.381s] [ext: omni.isaac.utils-0.6.0] startup
[5.387s] [ext: omni.isaac.wheeled_robots-0.9.0] startup
[5.398s] [ext: omni.isaac.gym-0.10.0] startup
[5.399s] [ext: omni.kit.selection-0.1.3] startup
[5.400s] [ext: omni.kit.menu.common-1.1.4] startup
[5.401s] [ext: omni.kit.widget.layers-1.6.32] startup
[5.407s] [ext: omni.kit.menu.edit-1.1.14] startup
[5.409s] [ext: omni.kit.menu.file-1.1.7] startup
[5.411s] [ext: omni.kit.property.layer-1.1.5] startup
[5.412s] [ext: semantics.schema.editor-0.3.4] startup
[5.414s] [ext: omni.kit.property.camera-1.0.5] startup
[5.415s] [ext: omni.kit.property.audio-1.0.9] startup
[5.418s] [ext: omni.kit.profiler.window-2.1.2] startup
2023-12-28 01:04:09 [5,479ms] [Warning] [omni.kit.profiler.window] remove _SpanInstance.__lt__ and use insort 'key' arg instead
[5.490s] [ext: omni.kit.widget.stage_icons-1.0.4] startup
[5.491s] [ext: omni.hydra.scene_api-0.1.2] startup
[5.497s] [ext: omni.kit.property.light-1.0.8] startup
[5.499s] [ext: omni.kit.property.skel-1.0.4] startup
[5.500s] [ext: omni.kit.window.stage-2.4.4] startup
[5.504s] [ext: omni.kit.property.transform-1.3.10] startup
[5.506s] [ext: omni.kit.property.render-1.1.1] startup
[5.507s] [ext: omni.kit.property.material-1.8.36] startup
[5.510s] [ext: omni.kit.property.geometry-1.2.11] startup
[5.512s] [ext: omni.isaac.debug_draw-0.6.1] startup
2023-12-28 01:04:09 [5,507ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2023-12-28 01:04:09 [5,507ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
[5.518s] [ext: omni.kit.menu.stage-1.2.3] startup
[5.520s] [ext: omni.kit.property.bundle-1.2.9] startup
[5.521s] [ext: omni.kit.property.isaac-0.2.2] startup
[5.523s] [ext: omni.isaac.occupancy_map-0.3.2] startup
[5.538s] [ext: omni.isaac.mjcf-1.0.0] startup
2023-12-28 01:04:09 [5,529ms] [Warning] [omni.isaac.mjcf] The extension omni.isaac.mjcf has been deprecated. Please use omni.importer.mjcf instead.
[5.538s] [ext: omni.kit.viewport.actions-105.1.6] startup
[5.541s] [ext: omni.kit.widget.toolbar-1.5.6] startup
[5.545s] [ext: omni.kit.raycast.query-1.0.1] startup
[5.552s] [ext: omni.kit.manipulator.transform-104.6.15] startup
[5.553s] [ext: omni.rtx.window.settings-0.6.9] startup
[5.556s] [ext: omni.kit.viewport.menubar.display-105.1.0] startup
[5.558s] [ext: omni.kit.manipulator.tool.snap-1.4.2] startup
[5.562s] [ext: omni.hydra.pxr.settings-1.0.5] startup
[5.563s] [ext: omni.activity.pump-1.0.0] startup
[5.564s] [ext: omni.physx.commands-105.1.12-5.1] startup
[5.566s] [ext: omni.usdphysics.ui-105.1.12-5.1] startup
[5.588s] [ext: omni.kit.widget.live-2.1.6] startup
[5.589s] [ext: omni.kit.viewport.pxr-104.0.1] startup
[5.589s] [ext: omni.kit.viewport.ready-1.0.5] startup
[5.590s] [ext: omni.kit.viewport.rtx-104.0.0] startup
[5.590s] [ext: omni.physx.ui-105.1.12-5.1] startup
[5.609s] [ext: omni.kit.widget.cache_indicator-2.0.8] startup
[5.611s] [ext: usdrt.gf-1.0.2] startup
[5.611s] [ext: omni.kit.manipulator.selector-1.0.1] startup
[5.612s] [ext: omni.kit.window.commands-0.2.4] startup
[5.613s] [ext: omni.kit.window.console-0.2.10] startup
[5.621s] [ext: omni.kit.property.physx-0.1.0] startup
[5.661s] [ext: omni.kit.actions.window-1.1.0] startup
[5.663s] [ext: omni.physx.demos-105.1.12-5.1] startup
[5.666s] [ext: omni.kit.manipulator.prim-105.0.11] startup
[5.669s] [ext: omni.kit.window.toolbar-1.5.4] startup
[5.671s] [ext: omni.kit.manipulator.selection-104.0.8] startup
[5.672s] [ext: omni.kit.window.script_editor-1.7.4] startup
[5.674s] [ext: omni.kit.hotkeys.window-1.4.1] startup
[5.677s] [ext: omni.physx.camera-105.1.12-5.1] startup
[5.683s] [ext: omni.physx.cct-105.1.12-5.1] startup
[5.693s] [ext: omni.physx.graph-105.1.12-5.1] startup
[5.701s] [ext: omni.physx.supportui-105.1.12-5.1] startup
[5.711s] [ext: omni.physx.vehicle-105.1.12-5.1] startup
[5.719s] [ext: omni.kit.stage_column.payload-1.0.12] startup
[5.721s] [ext: omni.graph.ui_nodes-1.5.10] startup
[5.740s] [ext: omni.sensors.nv.common-1000.0.0-isaacsim.2023.1.1] startup
[5.746s] [ext: omni.sensors.nv.materials-1000.0.0-isaacsim.2023.1.1] startup
[5.747s] [ext: omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1] startup
2023-12-28 01:04:09 [5,739ms] [Error] [carb] [Plugin: libnvwpm2_factory.plugin.so] Could not load the dynamic library from /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm2_factory.plugin.so. Error: libnvrtc.so.11.2: cannot open shared object file: No such file or directory (Additional information may be available by running the process with the LD_DEBUG environment variable set)
2023-12-28 01:04:09 [5,739ms] [Warning] [carb] Potential plugin preload failed: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm2_factory.plugin.so
2023-12-28 01:04:09 [5,739ms] [Error] [omni.ext.plugin] [ext: omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1] failed to load native plugin: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm2_factory.plugin.so
2023-12-28 01:04:09 [5,739ms] [Error] [carb] [Plugin: libnvwpm_factory.plugin.so] Could not load the dynamic library from /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm_factory.plugin.so. Error: libnvrtc.so.11.2: cannot open shared object file: No such file or directory (Additional information may be available by running the process with the LD_DEBUG environment variable set)
2023-12-28 01:04:09 [5,739ms] [Warning] [carb] Potential plugin preload failed: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm_factory.plugin.so
2023-12-28 01:04:09 [5,739ms] [Error] [omni.ext.plugin] [ext: omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1] failed to load native plugin: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm_factory.plugin.so
[5.748s] [ext: omni.kit.window.status_bar-0.1.6] startup
[5.751s] [ext: omni.physx.bundle-105.1.12-5.1] startup
[5.752s] [ext: omni.graph.bundle.action-2.0.4] startup
[5.752s] [ext: omni.isaac.range_sensor-2.0.2] startup
[5.781s] [ext: omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1] startup
2023-12-28 01:04:09 [5,776ms] [Error] [carb] [Plugin: libomni.sensors.nv.lidar.rtxmodel_lidar_core.plugin.so] Could not load the dynamic library from /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1+lx64.r.cp310/bin/libomni.sensors.nv.lidar.rtxmodel_lidar_core.plugin.so. Error: libnvrtc.so.11.2: cannot open shared object file: No such file or directory (Additional information may be available by running the process with the LD_DEBUG environment variable set)
2023-12-28 01:04:09 [5,776ms] [Warning] [carb] Potential plugin preload failed: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1+lx64.r.cp310/bin/libomni.sensors.nv.lidar.rtxmodel_lidar_core.plugin.so
2023-12-28 01:04:09 [5,776ms] [Error] [omni.ext.plugin] [ext: omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1] failed to load native plugin: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1+lx64.r.cp310/bin/libomni.sensors.nv.lidar.rtxmodel_lidar_core.plugin.so
[5.785s] [ext: omni.sensors.nv.radar-1000.0.0-isaacsim.2023.1.1] startup
[5.789s] [ext: omni.kit.window.title-1.1.3] startup
[5.790s] [ext: omni.replicator.composer-1.2.17] startup
[5.802s] [ext: omni.replicator.isaac-1.11.3] startup
[5.818s] [ext: omni.kit.manipulator.camera-105.0.4] startup
[5.820s] [ext: omni.kit.viewport.menubar.camera-105.1.9] startup
[5.823s] [ext: omni.isaac.sensor-9.11.1] startup
[5.870s] [ext: omni.kit.viewport.menubar.render-105.1.1] startup
[5.872s] [ext: omni.kit.viewport.menubar.settings-105.0.9] startup
[5.875s] [ext: omni.replicator.replicator_yaml-2.0.4] startup
[5.881s] [ext: omni.rtx.settings.core-0.5.13] startup
[5.884s] [ext: omni.kit.stage_column.variant-1.0.13] startup
[5.885s] [ext: omni.isaac.quadruped-1.4.1] startup
[5.893s] [ext: semantics.schema.property-1.0.2] startup
[5.894s] [ext: omni.kit.viewport.bundle-104.0.1] startup
[5.895s] [ext: omni.isaac.scene_blox-0.1.0] startup
[5.896s] [ext: omni.importer.urdf-1.6.1] startup
[5.925s] [ext: omni.importer.mjcf-1.1.0] startup
[5.931s] [ext: omni.kit.window.stats-0.1.4] startup
[5.932s] [ext: omni.isaac.sim.python-2023.1.1] startup
[5.932s] Simulation App Starting
2023-12-28 01:04:10 [5,939ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/ecoMode/enabled has been requested but is not available
2023-12-28 01:04:10 [5,939ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/directLighting/enabled has been requested but is not available
2023-12-28 01:04:10 [5,939ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/reflections/enabled has been requested but is not available
2023-12-28 01:04:10 [5,939ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/translucency/enabled has been requested but is not available
2023-12-28 01:04:10 [5,939ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/raytracing/subsurface/enabled has been requested but is not available
2023-12-28 01:04:10 [5,939ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/caustics/enabled has been requested but is not available
2023-12-28 01:04:11 [6,910ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-28 01:04:11 [6,910ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[7.607s] app ready
2023-12-28 01:04:12 [7,943ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-28 01:04:12 [7,943ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[8.134s] RTX ready
[8.275s] Simulation App Startup Complete
2023-12-28 01:04:13 [9,106ms] [Warning] [omni.importer.urdf] The path wrist_3-flange is not a valid usd path, modifying to wrist_3_flange
2023-12-28 01:04:13 [9,106ms] [Warning] [omni.importer.urdf] The path flange-tool0 is not a valid usd path, modifying to flange_tool0
2023-12-28 01:04:13 [9,106ms] [Warning] [omni.importer.urdf] The path wrist_3-flange is not a valid usd path, modifying to wrist_3_flange
2023-12-28 01:04:13 [9,106ms] [Warning] [omni.importer.urdf] The path flange-tool0 is not a valid usd path, modifying to flange_tool0
2023-12-28 01:04:13 [9,108ms] [Warning] [omni.importer.urdf] Link base_fixture_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:04:13 [9,109ms] [Warning] [omni.importer.urdf] Link base_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:04:13 [9,188ms] [Warning] [omni.importer.urdf] Link flange has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:04:13 [9,189ms] [Warning] [omni.importer.urdf] Link tool0 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:04:13 [9,191ms] [Warning] [omni.importer.urdf] Link base_link_1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:04:13 [9,265ms] [Warning] [omni.importer.urdf] Link flange_1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:04:13 [9,266ms] [Warning] [omni.importer.urdf] Link tool1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
Traceback (most recent call last):
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 354, in <module>
    main()
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 118, in main
    robot, robot_prim_path = add_robot_to_scene(robot_cfg, my_world)
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/helper.py", line 115, in add_robot_to_scene
    robot_p.set_solver_velocity_iteration_count(4)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation.py", line 981, in set_solver_velocity_iteration_count
    self._articulation_view.set_solver_velocity_iteration_counts(count)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation_view.py", line 3243, in set_solver_velocity_iteration_counts
    set_prim_property(self.prim_paths[i], "physxArticulation:solverVelocityIterationCount", counts[read_idx])
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/prims.py", line 802, in set_prim_property
    prim.GetAttribute(property_name).Set(property_value)
pxr.Tf.ErrorException: 
    Error in 'pxrInternal_v0_22__pxrReserved__::UsdStage::_SetValueImpl' at line 6174 in file /buildAgent/work/ac88d7d902b57417/USD/pxr/usd/usd/stage.cpp : 'Empty typeName for </World/ur10e_robot.physxArticulation:solverVelocityIterationCount>'
2023-12-28 01:04:13 [9,279ms] [Warning] [carb] [Plugin: omni.spectree.delegate.plugin] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.usd_resolver/bin/libomni.spectree.delegate.plugin.so remained loaded after unload request
2023-12-28 01:04:13 [9,281ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:04:13 [9,282ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal
2023-12-28 01:04:13 [9,294ms] [Warning] [carb.audio.context] 1 contexts were leaked
2023-12-28 01:04:13 [9,349ms] [Warning] [carb] Recursive unloadAllPlugins() detected!
2023-12-28 01:04:13 [9,361ms] [Warning] [omni.core.ITypeFactory] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.activity.core/bin/libomni.activity.core.plugin.so remained loaded after unload request.
eai@eai-Z790-UD-AX:~/rp/build_curobo/for_isaac_sim/curobo$

After I commented out the Line 115 and Line 116, Isaac Sim could start normally and load the robot arm (omni_python examples/isaac_sim/multi_arm_reacher.py --robot dual_ur10e.yml), but the Isaac Sim crashed after clicking the play button.

[7.482s] app ready
2023-12-28 01:08:12 [7,816ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-28 01:08:12 [7,816ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[7.952s] RTX ready
[8.180s] Simulation App Startup Complete
2023-12-28 01:08:13 [9,014ms] [Warning] [omni.importer.urdf] The path wrist_3-flange is not a valid usd path, modifying to wrist_3_flange
2023-12-28 01:08:13 [9,014ms] [Warning] [omni.importer.urdf] The path flange-tool0 is not a valid usd path, modifying to flange_tool0
2023-12-28 01:08:13 [9,014ms] [Warning] [omni.importer.urdf] The path wrist_3-flange is not a valid usd path, modifying to wrist_3_flange
2023-12-28 01:08:13 [9,014ms] [Warning] [omni.importer.urdf] The path flange-tool0 is not a valid usd path, modifying to flange_tool0
2023-12-28 01:08:13 [9,015ms] [Warning] [omni.importer.urdf] Link base_fixture_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:08:13 [9,016ms] [Warning] [omni.importer.urdf] Link base_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:08:13 [9,095ms] [Warning] [omni.importer.urdf] Link flange has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:08:13 [9,096ms] [Warning] [omni.importer.urdf] Link tool0 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:08:13 [9,097ms] [Warning] [omni.importer.urdf] Link base_link_1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:08:13 [9,169ms] [Warning] [omni.importer.urdf] Link flange_1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:08:13 [9,171ms] [Warning] [omni.importer.urdf] Link tool1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
warming up...
Curobo is Ready
[15.392s] [ext: omni.kit.asset_converter-2.0.18] startup
[15.409s] [ext: omni.kit.tool.asset_importer-2.4.28] startup
[15.445s] [ext: omni.isaac.asset_browser-0.4.3] startup
2023-12-28 01:08:40 [35,828ms] [Warning] [omni.client.python] Detected a blocking function. This will cause hitches or hangs in the UI. Please switch to the async version:
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 354, in <module>
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 165, in main
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/scenes/scene.py", line 225, in add_default_ground_plane
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/nucleus.py", line 572, in get_assets_root_path
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/nucleus.py", line 192, in check_server
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/extscore/omni.client/omni/client/__init__.py", line 158, in stat

2023-12-28 01:10:33 [148,722ms] [Warning] [omni.hydra] Mesh '/World/obstacles/table_mesh' has corrupted data in primvar 'displayColor': buffer size 8 doesn't match expected size 36 in faceVarying primvars
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
2023-12-28 01:10:45 [161,003ms] [Warning] [omni.isaac.dynamic_control.plugin] Failed to find articulation at '/World/ur10e_robot'
2023-12-28 01:10:45 [161,003ms] [Error] [omni.isaac.dynamic_control.plugin] DcGetArticulationRootBody: Invalid or expired articulation handle
2023-12-28 01:10:45 [161,004ms] [Warning] [omni.physx.tensors.plugin] Failed to find articulation at '/World/ur10e_robot'
2023-12-28 01:10:45 [161,004ms] [Error] [omni.physx.tensors.plugin] Pattern '/World/ur10e_robot' did not match any articulations

Traceback (most recent call last):
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 354, in <module>
    main()
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 219, in main
    my_world.reset()
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/world/world.py", line 408, in reset
    self._scene._finalize(self.physics_sim_view)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/scenes/scene.py", line 322, in _finalize
    robot.initialize(physics_sim_view)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation.py", line 294, in initialize
    self._articulation_view.initialize(physics_sim_view=physics_sim_view)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation_view.py", line 349, in initialize
    assert self._physics_view.is_homogeneous
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extsPhysics/omni.physics.tensors-105.1.12-5.1/omni/physics/tensors/impl/api.py", line 170, in is_homogeneous
    return self._backend.is_homogeneous
AttributeError: 'NoneType' object has no attribute 'is_homogeneous'
2023-12-28 01:10:45 [161,012ms] [Warning] [carb] [Plugin: omni.spectree.delegate.plugin] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.usd_resolver/bin/libomni.spectree.delegate.plugin.so remained loaded after unload request
2023-12-28 01:10:45 [161,013ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:10:45 [161,014ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal
2023-12-28 01:10:45 [161,016ms] [Warning] [carb.audio.context] 1 contexts were leaked
2023-12-28 01:10:45 [161,039ms] [Warning] [carb] Recursive unloadAllPlugins() detected!
2023-12-28 01:10:45 [161,046ms] [Warning] [omni.core.ITypeFactory] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.activity.core/bin/libomni.activity.core.plugin.so remained loaded after unload request.
eai@eai-Z790-UD-AX:~/rp/build_curobo/for_isaac_sim/curobo$

I wanted to check if you installed usd-core which you should not. Can you uninstall usd-core?

I wanted to check if you installed usd-core which you should not. Can you uninstall usd-core?

Hi @balakumar-s , usd-core has been remove from Isaac Sim Python environment, but the problem is still the same (omni_python examples/isaac_sim/multi_arm_reacher.py --robot dual_ur10e.ym).

eai@eai-Z790-UD-AX:~/rp/build_curobo/for_isaac_sim/curobo$ omni_python -m pip uninstall usd-core
Found existing installation: usd-core 23.11
Uninstalling usd-core-23.11:
  Would remove:
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/pxr/*
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core-23.11.dist-info/*
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libboost_python310-4bf84c61.so.1.78.0
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libtbb-3671e415.so.2
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_ar-f362f49a.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_arch-1f71a0ba.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_gf-5f8a4fe0.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_js-0d9c36ea.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_kind-03377bb6.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_ndr-e96f1be7.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_pcp-fe277e2d.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_plug-d7ac7d2b.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_sdf-ec6b5caf.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_sdr-f9e9ed64.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_tf-74256c82.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_trace-03be69d7.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usd-fc6e088b.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdGeom-98925d03.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdHydra-32f39d72.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdLux-f018e4c3.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdMedia-19200e15.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdPhysics-feaf1a3b.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdProc-f9b0f2b2.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdRender-30f27dff.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdRi-01a85c2c.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdShade-09400a59.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdSkel-8529d519.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdUI-85a91a62.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdUtils-1cef7cf7.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_usdVol-d2f19872.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_vt-e5fa06d1.so
    /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/python/lib/python3.10/site-packages/usd_core.libs/libusd_work-feb657a8.so
Proceed (Y/n)? Y
  Successfully uninstalled usd-core-23.11

[5.582s] [ext: omni.kit.stage_column.payload-1.0.12] startup
[5.583s] [ext: omni.graph.ui_nodes-1.5.10] startup
[5.603s] [ext: omni.sensors.nv.common-1000.0.0-isaacsim.2023.1.1] startup
[5.609s] [ext: omni.sensors.nv.materials-1000.0.0-isaacsim.2023.1.1] startup
[5.610s] [ext: omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1] startup
2023-12-28 01:20:27 [5,603ms] [Error] [carb] [Plugin: libnvwpm2_factory.plugin.so] Could not load the dynamic library from /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm2_factory.plugin.so. Error: libnvrtc.so.11.2: cannot open shared object file: No such file or directory (Additional information may be available by running the process with the LD_DEBUG environment variable set)
2023-12-28 01:20:27 [5,603ms] [Warning] [carb] Potential plugin preload failed: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm2_factory.plugin.so
2023-12-28 01:20:27 [5,603ms] [Error] [omni.ext.plugin] [ext: omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1] failed to load native plugin: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm2_factory.plugin.so
2023-12-28 01:20:27 [5,603ms] [Error] [carb] [Plugin: libnvwpm_factory.plugin.so] Could not load the dynamic library from /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm_factory.plugin.so. Error: libnvrtc.so.11.2: cannot open shared object file: No such file or directory (Additional information may be available by running the process with the LD_DEBUG environment variable set)
2023-12-28 01:20:27 [5,603ms] [Warning] [carb] Potential plugin preload failed: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm_factory.plugin.so
2023-12-28 01:20:27 [5,603ms] [Error] [omni.ext.plugin] [ext: omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1] failed to load native plugin: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.wpm-1000.0.0-isaacsim.2023.1.1+lx64.r/bin/libnvwpm_factory.plugin.so
[5.611s] [ext: omni.kit.window.status_bar-0.1.6] startup
[5.615s] [ext: omni.physx.bundle-105.1.12-5.1] startup
[5.615s] [ext: omni.graph.bundle.action-2.0.4] startup
[5.615s] [ext: omni.isaac.range_sensor-2.0.2] startup
[5.645s] [ext: omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1] startup
2023-12-28 01:20:27 [5,641ms] [Error] [carb] [Plugin: libomni.sensors.nv.lidar.rtxmodel_lidar_core.plugin.so] Could not load the dynamic library from /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1+lx64.r.cp310/bin/libomni.sensors.nv.lidar.rtxmodel_lidar_core.plugin.so. Error: libnvrtc.so.11.2: cannot open shared object file: No such file or directory (Additional information may be available by running the process with the LD_DEBUG environment variable set)
2023-12-28 01:20:27 [5,641ms] [Warning] [carb] Potential plugin preload failed: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1+lx64.r.cp310/bin/libomni.sensors.nv.lidar.rtxmodel_lidar_core.plugin.so
2023-12-28 01:20:27 [5,641ms] [Error] [omni.ext.plugin] [ext: omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1] failed to load native plugin: /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extscache/omni.sensors.nv.lidar-1000.0.1-isaacsim.2023.1.1+lx64.r.cp310/bin/libomni.sensors.nv.lidar.rtxmodel_lidar_core.plugin.so
[5.649s] [ext: omni.sensors.nv.radar-1000.0.0-isaacsim.2023.1.1] startup
[5.653s] [ext: omni.kit.window.title-1.1.3] startup
[5.654s] [ext: omni.replicator.composer-1.2.17] startup
[5.666s] [ext: omni.replicator.isaac-1.11.3] startup
[5.683s] [ext: omni.kit.manipulator.camera-105.0.4] startup
[5.685s] [ext: omni.kit.viewport.menubar.camera-105.1.9] startup
[5.689s] [ext: omni.isaac.sensor-9.11.1] startup
[5.736s] [ext: omni.kit.viewport.menubar.render-105.1.1] startup
[5.738s] [ext: omni.kit.viewport.menubar.settings-105.0.9] startup
[5.741s] [ext: omni.replicator.replicator_yaml-2.0.4] startup
[5.747s] [ext: omni.rtx.settings.core-0.5.13] startup
[5.750s] [ext: omni.kit.stage_column.variant-1.0.13] startup
[5.751s] [ext: omni.isaac.quadruped-1.4.1] startup
[5.759s] [ext: semantics.schema.property-1.0.2] startup
[5.761s] [ext: omni.kit.viewport.bundle-104.0.1] startup
[5.761s] [ext: omni.isaac.scene_blox-0.1.0] startup
[5.762s] [ext: omni.importer.urdf-1.6.1] startup
[5.791s] [ext: omni.importer.mjcf-1.1.0] startup
[5.797s] [ext: omni.kit.window.stats-0.1.4] startup
[5.798s] [ext: omni.isaac.sim.python-2023.1.1] startup
[5.799s] Simulation App Starting
2023-12-28 01:20:27 [5,806ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/ecoMode/enabled has been requested but is not available
2023-12-28 01:20:27 [5,807ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/directLighting/enabled has been requested but is not available
2023-12-28 01:20:27 [5,807ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/reflections/enabled has been requested but is not available
2023-12-28 01:20:27 [5,807ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/translucency/enabled has been requested but is not available
2023-12-28 01:20:27 [5,807ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/raytracing/subsurface/enabled has been requested but is not available
2023-12-28 01:20:27 [5,807ms] [Warning] [omni.kit.widget.settings.settings_model] a value for setting /rtx/caustics/enabled has been requested but is not available
2023-12-28 01:20:28 [6,758ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-28 01:20:28 [6,758ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[7.466s] app ready
2023-12-28 01:20:29 [7,823ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2023-12-28 01:20:29 [7,823ms] [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
[7.946s] RTX ready
[8.178s] Simulation App Startup Complete
2023-12-28 01:20:30 [9,007ms] [Warning] [omni.importer.urdf] The path wrist_3-flange is not a valid usd path, modifying to wrist_3_flange
2023-12-28 01:20:30 [9,007ms] [Warning] [omni.importer.urdf] The path flange-tool0 is not a valid usd path, modifying to flange_tool0
2023-12-28 01:20:30 [9,007ms] [Warning] [omni.importer.urdf] The path wrist_3-flange is not a valid usd path, modifying to wrist_3_flange
2023-12-28 01:20:30 [9,007ms] [Warning] [omni.importer.urdf] The path flange-tool0 is not a valid usd path, modifying to flange_tool0
2023-12-28 01:20:30 [9,008ms] [Warning] [omni.importer.urdf] Link base_fixture_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:20:30 [9,010ms] [Warning] [omni.importer.urdf] Link base_link has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:20:30 [9,090ms] [Warning] [omni.importer.urdf] Link flange has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:20:30 [9,091ms] [Warning] [omni.importer.urdf] Link tool0 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:20:30 [9,093ms] [Warning] [omni.importer.urdf] Link base_link_1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:20:30 [9,167ms] [Warning] [omni.importer.urdf] Link flange_1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
2023-12-28 01:20:30 [9,168ms] [Warning] [omni.importer.urdf] Link tool1 has no colliders, and no inertia was imported; assigning a small isotropic inertia matrix
warming up...
Curobo is Ready
[15.194s] [ext: omni.kit.asset_converter-2.0.18] startup
[15.210s] [ext: omni.kit.tool.asset_importer-2.4.28] startup
[15.247s] [ext: omni.isaac.asset_browser-0.4.3] startup
2023-12-28 01:20:57 [35,618ms] [Warning] [omni.client.python] Detected a blocking function. This will cause hitches or hangs in the UI. Please switch to the async version:
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 354, in <module>
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 165, in main
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/scenes/scene.py", line 225, in add_default_ground_plane
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/nucleus.py", line 572, in get_assets_root_path
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/utils/nucleus.py", line 192, in check_server
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/extscore/omni.client/omni/client/__init__.py", line 158, in stat

2023-12-28 01:22:49 [148,160ms] [Warning] [omni.hydra] Mesh '/World/obstacles/table_mesh' has corrupted data in primvar 'displayColor': buffer size 8 doesn't match expected size 36 in faceVarying primvars
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
**** Click Play to start simulation *****
2023-12-28 01:22:58 [156,617ms] [Warning] [omni.isaac.dynamic_control.plugin] Failed to find articulation at '/World/ur10e_robot'
2023-12-28 01:22:58 [156,617ms] [Error] [omni.isaac.dynamic_control.plugin] DcGetArticulationRootBody: Invalid or expired articulation handle
2023-12-28 01:22:58 [156,617ms] [Warning] [omni.physx.tensors.plugin] Failed to find articulation at '/World/ur10e_robot'
2023-12-28 01:22:58 [156,617ms] [Error] [omni.physx.tensors.plugin] Pattern '/World/ur10e_robot' did not match any articulations

Traceback (most recent call last):
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 354, in <module>
    main()
  File "/home/eai/rp/build_curobo/for_isaac_sim/curobo/examples/isaac_sim/multi_arm_reacher.py", line 219, in main
    my_world.reset()
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/world/world.py", line 408, in reset
    self._scene._finalize(self.physics_sim_view)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/scenes/scene.py", line 322, in _finalize
    robot.initialize(physics_sim_view)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation.py", line 294, in initialize
    self._articulation_view.initialize(physics_sim_view=physics_sim_view)
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/exts/omni.isaac.core/omni/isaac/core/articulations/articulation_view.py", line 349, in initialize
    assert self._physics_view.is_homogeneous
  File "/home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/extsPhysics/omni.physics.tensors-105.1.12-5.1/omni/physics/tensors/impl/api.py", line 170, in is_homogeneous
    return self._backend.is_homogeneous
AttributeError: 'NoneType' object has no attribute 'is_homogeneous'
2023-12-28 01:22:58 [156,626ms] [Warning] [carb] [Plugin: omni.spectree.delegate.plugin] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.usd_resolver/bin/libomni.spectree.delegate.plugin.so remained loaded after unload request
2023-12-28 01:22:58 [156,627ms] [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Annotators' for removal
2023-12-28 01:22:58 [156,628ms] [Warning] [omni.graph.core.plugin] Could not find category 'Replicator:Core' for removal
2023-12-28 01:22:58 [156,630ms] [Warning] [carb.audio.context] 1 contexts were leaked
2023-12-28 01:22:58 [156,668ms] [Warning] [carb] Recursive unloadAllPlugins() detected!
2023-12-28 01:22:58 [156,675ms] [Warning] [omni.core.ITypeFactory] Module /home/eai/.local/share/ov/pkg/isaac_sim-2023.1.1/kit/exts/omni.activity.core/bin/libomni.activity.core.plugin.so remained loaded after unload request.
eai@eai-Z790-UD-AX:~/rp/build_curobo/for_isaac_sim/curobo$

I don't know what's happening in your machine. We won't be able to test on Isaac Sim 2023.1.1 soon.

I would advise to use Isaac sim 2023.1.0 where we know for sure that everything works.

We also advise to use a docker following https://curobo.org/source/getting_started/5_docker_development.html as that helps us reproduce any errors you get.

NVlabs / curobo

nvrtc: error: invalid value for --gpu-architecture (-arch) #87

Issue 1

Issue 2

Pytest Error

Isaac Sim Error