search

space-ros / simulation

Simulation assets of space-ros demos
8 stars 19 forks source link

Reuse `space-ros/simulation` repository for IsaacSim #18

Open franklinselva opened 1 month ago

franklinselva commented 1 month ago

Since the contents of this repository are moved to space-ros/demos based on space-ros/space-ros#178, I believe this repository can be reused exclusively for Isaac Sim which leverages photorealistic simulations.

Feature Proposal

The proposal is to repurpose the current repository to manage the simualtion assets and extensions for Space ROS simulation with Nvidia's Isaac SIM. Eventually the repository will be repurposed as the extension pack of Space ROS provided for Nvidia's Isaac SIM. The assets are made photo-realistic with a minimal sim2real gap with the information provided in the internet resources. The assets are a compilation of large set of use cases related to Space ROS applications which is summarized below. In addition to the assets, new extensions are created for space ros demos which serve as an example for building an ecosystem of simualtion extensions which could grow over time. A detailed documentation of all the assets and extensions are provided to help with the usage of the extension module.

Environment Assets

  1. International Space Station Model. Useful for Canadarm2 related maintainence and docking applications. international-space-station canadarm2-iss

  2. Interior of International Space Station. Useful for Astrobee and Robonaut2 for assistance related applications. interior-of-international-space-station

  3. Mars Terrain. 800 sq.m area of realistic mars terrain data acquired from NASA's Mars Exploration Data. mars-environment-curiosity-rover

  4. Moon Terrain. 800 sq.m area of realistic moon terrain data acquired from NASA's Moon Exploraiton Data. moon-environment

  5. Simple Chess Envrionment for Robonaut2. Useful in manipulation and control related tasks for Robonaut2. robonaut2-chessboard-manipulation

  6. Simple Mars Replica for Ingenuity Helicopter. Useful for Visual Odometry, Search and Mission Planning related applidations. ingenuity-helicopter-simple

Robot Assets

  1. Canadarm2. The gazebosim equivalent of canadarm2 robot model. canadarm2
  2. Curiosity Mars Rover. The gazebosim equivalent of curiosity Mars rover. curiosity-rover
  3. Perseverance Mars Rover. The high fidelity Robot Model with advanced sensors for Mars exploration related tasks. perseverance-rover
  4. Robonaut2 Upper body. Manipulation tasks and planning for robonaut2 upper body. robonaut-upperbody
  5. Robonaut2 Full body. Useful for applications with Interior of ISS. robonaut
  6. Ingenuity Helicopter. Aerial robot for search applications in mars terrain. ingenuity-helicopter
  7. Canadarm2 Destro. The docking and maintainence robot for ISS. canadarm2-17dof

IsaacSim Extensions

  1. An SpaceROS specific Omnigraph extensions for controlling rovers with rocker-bogie mechanism
  2. An example omnigraph extensions for building new extension specific to Space ROS

Documentation

  1. Detailed description and information on Environment assets
  2. Detailed description and usage instructions on robot assets
  3. Detailed description and usage instructions on extensions

The assets are widely tested for its functionality and are mostly simulation ready at the moment. The demo packages are already being built for direct integration between gazebosim and isaac sim. These assets should help cover wide range of subdomains and features exptected to be built on space ROS project.

The potential directions moving forward from this feature is to create a standalone simulator built on top of Nvidia's Omniverse for better control in packaging and distributing the simulation assets and version controls of extensions.

MetaInfo

Title: Simulation Assets and Extensions for Nvidia's Isaac Sim Freelancer ID: @selvakumarhs NASA Space ROS SIM Challenge

franklinselva commented 1 week ago

@mkhansenbot @EzraBrooks I am trying to push the work on simulation assets for Nvidia's Isaac Sim. I am using git-lfs for pushing the large files to the repository but couln't succeed in pushing the changes to the fork.

I am thrown with,

batch response: @franklinselva can not upload new objects to public fork franklinselva/spaceros-simulation                                                                                                                                                                                                                                                                        
batch response: @franklinselva can not upload new objects to public fork franklinselva/spaceros-simulation
error: failed to push some refs to 'https://github.com/franklinselva/spaceros-simulation.git'

Upon looking on the issue related to this, the commit on git-lfs mentions the repository should already have lfs setup on the original repository or the contributor should have the push permissions for the original repository i.e. write access to space-ros/simulation. Can you please help me resolve this issue?

EzraBrooks commented 1 week ago

I'm not 100% sold on the idea that Isaac Sim, a proprietary simulator, should live in the same repository as non-proprietary simulators. I don't want Space ROS to trend toward being vendor-locked by NVIDIA. I don't even have a computer with an NVIDIA GPU 😉

franklinselva commented 1 week ago

One of the questions on the SpaceROS SIM Webinar was to choose Nvidia's Isaac Sim / Unreal engine for improved visual fidelity. The response to the question was we could go for one of the simulators in this case.

The Q&A of the webinar can be found here. Rows 3 and 27 are related to this issue.

I completely agree with the setup of requiring NVidia GPU for such simulators. For this purpose, I have already managed to write a workaround for this. More information can be found here.

Moving further, IsaacSim is not a definitive simulator but a reference application built on top of Nvidia Omniverse. This feature intends to come up with a dedicated application as I referred to in the description of this issue.

EzraBrooks commented 1 week ago

Gotcha. I was more just saying "maybe it doesn't live in the same place as the Gz simulation", not that it doesn't have a place at all. I totally understand the desire for higher-fidelity perception simulation.

I'll take a look at setting up LFS here, since that's a good improvement anyway.

franklinselva commented 1 week ago

Thanks a lot. I am currently setting up the LFS template for an easier entry point. I am stuck with pushing the assets to the repository at the moment. Is it something, I could overcome soon? The due date for the challenge is right around the corner.

EzraBrooks commented 1 week ago

See #26, #27

EzraBrooks commented 1 week ago

LFS should be enabled on the repo now. Let me know if it still gives you issues - maybe it's a permissions thing.

If it is, I'm not sure what the right route forward here is - giving upstream write access to everyone who wants to contribute doesn't seem sustainable.

franklinselva commented 1 week ago

LFS setup works good. I have already pushed the change. Thanks for the help!