search

btx0424 / OmniDrones

https://omnidrones.readthedocs.io/
MIT License
147 stars 25 forks source link

Can not locate "omni.isaac.orbit" #43

Open RuiqiZhang99 opened 7 months ago

RuiqiZhang99 commented 7 months ago

Hi, I meet an issue after install all the package, could you help me to solve it? Version: Ubuntu20.04 IsaacSim-2023.1.0-hotfix.1 Python-3.10 After I install all the packages as tutorial and run train.py with my wandb id, it prints the config information and then a bug comes:

[2024-02-07 23:18:13,254][wandb][DEBUG] - no default config file found in config-defaults.yaml
task:
  env:
    num_envs: 4096
    env_spacing: 8
    max_episode_length: 500
  sim:
    dt: 0.016
    substeps: 1
    gravity:
    - 0
    - 0
    - -9.81
    replicate_physics: false
    use_flatcache: true
    use_gpu_pipeline: true
    device: cuda:0
    solver_type: 1
    use_gpu: true
    bounce_threshold_velocity: 0.2
    friction_offset_threshold: 0.04
    friction_correlation_distance: 0.025
    enable_stabilization: true
    gpu_max_rigid_contact_count: 524288
    gpu_max_rigid_patch_count: 163840
    gpu_found_lost_pairs_capacity: 4194304
    gpu_found_lost_aggregate_pairs_capacity: 33554432
    gpu_total_aggregate_pairs_capacity: 4194304
    gpu_max_soft_body_contacts: 1048576
    gpu_max_particle_contacts: 1048576
    gpu_heap_capacity: 67108864
    gpu_temp_buffer_capacity: 16777216
    gpu_max_num_partitions: 8
  name: Hover
  drone_model: Firefly
  force_sensor: false
  time_encoding: true
  reward_effort_weight: 0.1
  reward_action_smoothness_weight: 0.0
  reward_motion_smoothness_weight: 0.0
  reward_distance_scale: 1.2
  action_transform: null
algo:
  name: ppo
  train_every: 32
  ppo_epochs: 4
  num_minibatches: 16
  priv_actor: false
  priv_critic: false
  checkpoint_path: null
headless: true
sim:
  dt: 0.016
  substeps: 1
  gravity:
  - 0
  - 0
  - -9.81
  replicate_physics: false
  use_flatcache: true
  use_gpu_pipeline: true
  device: cuda:0
  solver_type: 1
  use_gpu: true
  bounce_threshold_velocity: 0.2
  friction_offset_threshold: 0.04
  friction_correlation_distance: 0.025
  enable_stabilization: true
  gpu_max_rigid_contact_count: 524288
  gpu_max_rigid_patch_count: 163840
  gpu_found_lost_pairs_capacity: 4194304
  gpu_found_lost_aggregate_pairs_capacity: 33554432
  gpu_total_aggregate_pairs_capacity: 4194304
  gpu_max_soft_body_contacts: 1048576
  gpu_max_particle_contacts: 1048576
  gpu_heap_capacity: 67108864
  gpu_temp_buffer_capacity: 16777216
  gpu_max_num_partitions: 8
env:
  num_envs: 4096
  env_spacing: 8
  max_episode_length: 500
total_frames: 150000000
max_iters: -1
eval_interval: -1
save_interval: -1
seed: 0
viewer:
  resolution:
  - 960
  - 720
  eye:
  - 8
  - 0.0
  - 6.0
  lookat:
  - 0.0
  - 0.0
  - 1.0
wandb:
  group: Hover
  run_name: Hover-ppo
  job_type: train
  entity: afc30efa2dd33891947c0ec7035bbe2f085d9243
  project: omnidrones
  mode: disabled
  run_id: null
  monitor_gym: true
  tags: null

Error executing job with overrides: ['algo=ppo', 'wandb.entity=afc30efa2dd33891947c0ec7035bbe2f085d9243']
Traceback (most recent call last):
  File "/home/rich/OmniDrones/scripts/train.py", line 48, in main
    from omni_drones.envs.isaac_env import IsaacEnv
  File "/home/rich/OmniDrones/omni_drones/envs/__init__.py", line 32, in <module>
    from .pinball import Pinball
  File "/home/rich/OmniDrones/omni_drones/envs/pinball.py", line 43, in <module>
    from omni.isaac.orbit.sensors import ContactSensorCfg, ContactSensor
ModuleNotFoundError: No module named 'omni.isaac.orbit'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Is the omni.isaac.orbit an extension of isaacsim? Do we need to download and install it additionally? THANKS

btx0424 commented 7 months ago

Hi there. you can check out Isaac Orbit here. OmniDrones uses Orbit's contact sensors and ray-casting lidar implementation for Pinball and Forest. All other tasks do not need it.

In your case, just comment out the from .pinball import Pinball line.

RuiqiZhang99 commented 7 months ago

Thanks for the response. BTW, I'd like to know if there is any suggestions to solve the lack of GPU memory. (mine is RTX3070-8G) After I reduce the gpu_capacity related configs of demo.yaml and run demo_drone.py, it still comes that 

Please either pass the dim explicitly or simply use torch.linalg.cross.
The default value of dim will change to agree with that of linalg.cross in a future release. (Triggered internally at ../aten/src/ATen/native/Cross.cpp:63.)
  + torch.cross(ang_vel, ang_vel)
/home/rich/miniconda3/envs/od/lib/python3.10/site-packages/torch/_functorch/deprecated.py:61: UserWarning: We've integrated functorch into PyTorch. As the final step of the integration, functorch.vmap is deprecated as of PyTorch 2.0 and will be deleted in a future version of PyTorch >= 2.3. Please use torch.vmap instead; see the PyTorch 2.0 release notes and/or the torch.func migration guide for more details https://pytorch.org/docs/master/func.migrating.html
  warn_deprecated('vmap', 'torch.vmap')
2024-02-13 10:37:36 [75,206ms] [Error] [omni.syntheticdata.plugin] OgnSdRenderVarPtr : invalid render product.
2024-02-13 10:37:36 [75,206ms] [Error] [omni.syntheticdata.plugin] OgnSdRenderVarPtr : invalid render product.
2024-02-13 10:37:36 [75,206ms] [Error] [omni.syntheticdata.plugin] OgnSdRenderVarPtr : invalid render product.
2024-02-13 10:37:36 [75,206ms] [Error] [omni.syntheticdata.plugin] OgnSdRenderVarPtr : invalid render product.
2024-02-13 10:37:36 [75,206ms] [Error] [omni.syntheticdata.plugin] OgnSdRenderVarPtr : invalid render product.
2024-02-13 10:37:36 [75,236ms] [Error] [carb.graphics-vulkan.plugin] VkResult: ERROR_OUT_OF_DEVICE_MEMORY
2024-02-13 10:37:36 [75,236ms] [Error] [carb.graphics-vulkan.plugin] vkAllocateMemory failed for flags: 0.
2024-02-13 10:37:36 [75,236ms] [Error] [gpu.foundation.plugin] Unable to allocate buffer
2024-02-13 10:37:36 [75,236ms] [Error] [gpu.foundation.plugin] Buffer creation failed for the device: 0.
2024-02-13 10:37:36 [75,236ms] [Error] [rtx.raytracing.plugin] Failed to create buffers for picking results!
2024-02-13 10:37:36 [75,240ms] [Error] [carb.graphics-vulkan.plugin] VkResult: ERROR_OUT_OF_DEVICE_MEMORY
2024-02-13 10:37:36 [75,240ms] [Error] [carb.graphics-vulkan.plugin] vkAllocateMemory failed for flags: 0.
2024-02-13 10:37:36 [75,240ms] [Error] [gpu.foundation.plugin] Unable to allocate buffer
2024-02-13 10:37:36 [75,240ms] [Error] [gpu.foundation.plugin] Buffer creation failed for the device: 0.
2024-02-13 10:37:36 [75,240ms] [Error] [rtx.raytracing.plugin] Failed to create buffers for picking results!
2024-02-13 10:37:36 [75,243ms] [Error] [carb.graphics-vulkan.plugin] VkResult: ERROR_OUT_OF_DEVICE_MEMORY
2024-02-13 10:37:36 [75,243ms] [Error] [carb.graphics-vulkan.plugin] vkAllocateMemory failed for flags: 0.
2024-02-13 10:37:36 [75,243ms] [Error] [gpu.foundation.plugin] Unable to allocate buffer
2024-02-13 10:37:36 [75,243ms] [Error] [gpu.foundation.plugin] Buffer creation failed for the device: 0.
  0%|                                                                                                                        | 0/1000 [00:00<?, ?it/s]
Error executing job with overrides: []

with the config

[8.881s] Simulation App Startup Complete
headless: false
sim:
  dt: 0.016
  substeps: 1
  gravity:
  - 0
  - 0
  - -9.81
  replicate_physics: false
  use_flatcache: true
  use_gpu_pipeline: true
  device: cuda:0
  solver_type: 1
  use_gpu: true
  bounce_threshold_velocity: 0.2
  friction_offset_threshold: 0.04
  friction_correlation_distance: 0.025
  enable_stabilization: true
  gpu_max_rigid_contact_count: 131072
  gpu_max_rigid_patch_count: 81920
  gpu_found_lost_pairs_capacity: 262144
  gpu_found_lost_aggregate_pairs_capacity: 524288
  gpu_total_aggregate_pairs_capacity: 524288
  gpu_max_soft_body_contacts: 262144
  gpu_max_particle_contacts: 262144
  gpu_heap_capacity: 2097152
  gpu_temp_buffer_capacity: 2097152
  gpu_max_num_partitions: 1
viewer:
  resolution:
  - 96
  - 72
  eye:
  - 2.0
  - 2.0
  - 2.0
  lookat:
  - 0.0
  - 0.0
  - 1.0
drone_model: Hummingbird
steps: 1000
btx0424 commented 7 months ago

Since this is not a typical PyTorch memory error, I wonder if you could train with a smaller number of environments, say, 1024.

I will add an option that disables Isaac Sim's rendering functionalities, which would save about 1G~2G GPU memory. Since this option can not be altered on-the-fly, the drawback is that you have to manually run another script (after training is finished) with rendering on to see the visuals.

Also, the previous issue you mentioned has been fixed in #46 .

btx0424 commented 7 months ago

By the way, noticing that you are from Berkeley HiPeR Lab, I want to say your previous work on training an adaptive drone controller was interesting. And I am currently working on something similar. May I kindly ask what you are planning to do with this repo?