[Proposal] Anyone tried adding an IMU Sensor?

[kennetms]

Proposal

There is already an IMU implemented in IsaacSim documented here https://docs.omniverse.nvidia.com/isaacsim/latest/features/sensors_simulation/isaac_sim_sensors_physics_based_imu.html.

Are there any plans to add an IMU Sensor to Orbit or has anyone already done this?

Motivation

Robotics tasks such as navigation can benefit from IMU Sensor data which is currently not implemented in Orbit.

Checklist

• [x] I have checked that there is no similar issue in the repo (required)

Acceptance Criteria

• [ ] Create an IMU Sensor under omni/isaac/orbit/sensors similar to current sensors such as camera and ray caster.
• [ ] Create an example use of the IMU Sensor under source/standalone/tutorials/04_sensors

[Ltesfaye]

I've been able to add and use omni.isaac.sensor.IMUSensor without much difficulty.

After spawning your environment and loading the scene, I was able to import the sensor using from omni.isaac.sensor import IMUSensor and spawn it using a method like the one below. Note that if you import the IMUSensor in the beginning, this can throw an error, and you need to import it in the spawn method or after the Environment is created. This is definitely not the cleanest way of doing it, but it works for my current purposes.

`

        oreintation = [1,0,0,0] if not self.imu_pointing_down else [ 0, 0, -1, 0]
        imu_path_addon = '' if imu_at_base else '/imu_link'
        self.imu_sensor = IMUSensor(
            prim_path=f"{robot_base_path}{imu_path_addon}/imu_sensor",
            name="imu",
            frequency=self.imu_frequency,
            translation=np.array([0, 0, 0]),
            orientation=np.array(oreintation), # rotating it to have z point down
            # linear_acceleration_filter_size = 10,
            # angular_velocity_filter_size = 10,
            # orientation_filter_size = 10,
        )
        self.imu_sensor.initialize(physics_sim_view) # please work
        self.imu_sensor.resume()

`

[Mayankm96]

Is the IMUSensor from Isaac Sim any different than just querying the rigid state from the links and computing them in the right frame? Also, do you know if it works for parallelized simulation?

We have an IMU sensor implementation that does the latter. But it didn't seem like it will be useful without a proper IMU noise model, etc.

[abizovnuralem]

I have tried to put the IMU sensor from the examples, but in my case, it doesn't work, I got zero values every time, with False in reading.is_valid. The example of code:

def update_imu(dt, imu_sensor, timeline, meters_per_unit, sensor_prim): if timeline.is_playing(): reading = imu_sensor.get_sensor_reading(sensor_prim) print(reading.is_valid)

    a = (reading.lin_acc_x) * meters_per_unit
    b = (reading.lin_acc_y) * meters_per_unit
    c = (reading.lin_acc_z) * meters_per_unit
    aa = (reading.ang_vel_x)
    bb = (reading.ang_vel_y)
    cc = (reading.ang_vel_z)
    tt = (reading.orientation[0])
    dd = (reading.orientation[1])
    yt = (reading.orientation[2])
    s = (reading.orientation[3])

Who got something different than zeros? Need help. Thanks.

[sesem738]

Has anyone been able to figure out the IMU situation?

[Ltesfaye]

Have you added the physics sim view to your IMU instance?@abizovnuralem

If not, that may be the reason. I was having a similar reading in the beginning, but that seemed to fix it. This is the same physics sim view as in the environment.

I have my responses to the earlier questions below @Mayankm96

Is the IMUSensor from Isaac Sim any different than just querying the rigid state from the links and computing them in the right frame?

As far as I can tell, the readings for the rigid state link and IMUSensor are the same during the render step. I didn't check this within a physics callback. I opted to use the IMUSensor naively because I got IMU readings to publish to a ros topic during each physics step with correct reference frames from the sensor readings.

Also, do you know if it works for parallelized simulation?

I have not tested this for parallel simulations. I am currently working on a single robot simulation in a large environment context. I can try testing this out next week and let you know if this approach scales.

[shaoxiang]

I used fromomni.isaac.sensor import IMUSensor this method, in the above mentionedand by @Ltesfaye https://github.com/isaac-sim/IsaacLab/issues/440#issuecomment-2142645565 . I can get the imu data, but the bad thing is that ang_vel data is all 0.0

imu_data: {‘time’: 6.690000057220459, ‘physics_step’: 670.0, ‘lin_acc’: tensor([6.0603, 4.2792, 6.4186], device=‘cuda:0’), ‘ang_vel’: tensor([0., 0., 0.], device=‘cuda:0’), ‘orientation’: tensor([-0.5061, 0.1487, 0.3882, 0.7556], device=‘cuda:0’)} The data is in pytorch tensor format and is in cuda: 0 device.

I also got the error as follows:

2024-07-09 06:05:07 [217,200ms] [Error] [omni.physx.tensors.plugin] Incompatible device of velocity tensor in function omni::physx::tensors::GpuRigidBodyView::getVelocities: expected device 0, received device -1

Hope to get help...