search

isaac-sim / IsaacLab

Unified framework for robot learning built on NVIDIA Isaac Sim
https://isaac-sim.github.io/IsaacLab
Other
1.9k stars 727 forks source link

[Proposal] Isaac Lab needs the concept of a true SENSOR class #464

Closed joehays closed 3 months ago

joehays commented 3 months ago

Proposal

Proprioceptive sensors in orbit are pulling information directly from the Articulated objects in Isaac Sim. There is no concept of a true SENSOR base class in Isaac Lab (or Orbit). The current modest set of sensors are only exteroceptive (like cameras). It doesn't apply to (e.g. IMU, joint torques with strain-guage non-idealities, quantized joint positions with bias, etc... Sensors have dynamics, non-idealities, biases, and other inaccuracies. Pulling obs directly from the physics engine doesn't allow any of these non-idealities to be modeled in a sensor object native to IsaacLab. IsaacLab needs to provide a base class sensor model that takes raw/exact data from IsaacSim and makes it available for derived sensors to be constructed by the user.

jtigue-bdai commented 3 months ago

Correct Isaac Lab does not currently handle real world affects on sensors. This is something on our minds and is being worked into the roadmap.

jtigue-bdai commented 3 months ago

One thing to also note is that Observations can be modified with Noise, clipping, and scaling. See: ObservationTermCfg. This isn't everything that you are looking for but is in that direction.