IMU data stream (Python)

[tRosenflanz]

---

Required Info
Camera Model	D435i
Firmware Version	05.10.13
Operating System & Version	Linux (Ubuntu 18.04)
Kernel Version (Linux Only)	4.15.0-45-generic
Platform	PC
SDK Version	2.16
Language	python
Segment	others

Issue Description

Hello, is there any code example for python to enable IMU stream for D435i camera?

I am currently trying this:

import pyrealsense2 as rs

def initialize_camera():
    #start the frames pipe
    p = rs.pipeline()
    conf = rs.config()
    conf.enable_stream(rs.stream.accel,rs.format.motion_xyz32f,200)
    conf.enable_stream(rs.stream.gyro,rs.format.motion_xyz32f,200)
    prof = p.start(conf)
    return p

p = initialize_camera()
f = p.wait_for_frames()
f.as_motion_frame().get_motion_data()

But it errors out with:

RuntimeError: null pointer passed for argument "frame_ref"

Which indicates that the IMU streams were not enabled.

I checked the calibration script but it seems to be going through the sensor directly and I was hoping this is not necessary in my case... Any help or direction to script would be appreciated!

P.S. I know the data can be collected since realsense-viewer loads the bag with recorded motion data

[tRosenflanz]

also tried:

    conf.enable_stream(rs.stream.accel,rs.format.motion_xyz32f)
    conf.enable_stream(rs.stream.gyro,rs.format.motion_xyz32f)

and

    conf.enable_stream(rs.stream.accel)
    conf.enable_stream(rs.stream.gyro)

with exactly the same results

[ev-mp]

There seem to be something wrong about those two lines:

cam = initialize_camera() f = p.wait_for_frames()

cam and p are not related

Also check

conf.enable_stream(rs.stream.accel,rs.format.motion_xyz32f,200) conf.enable_stream(rs.stream.gyro,rs.format.motion_xyz32f,200)

The accel stream for D435i supports either 63 or 250 fps rates

[tRosenflanz]

cam and p was just a typo here. In the code the pipeline is called p throughout. I changed the question to be consistent.

Tried FPS 250 for the accel, same error still. Worth noting that I can retrieve the colored frame from the same pipeline so it works correctly for other types of frames

[dorodnic]

I'm concerned about this line: f.as_motion_frame() f is of type frameset in other words collection of frames. I think as_motion_frame would only work if f is of size == 1 and contains singe motion frame. Did you try to iterate over f?

[tRosenflanz]

I have not. The rest of the frames have nice API of get_x_frame but not the motion data. Can I just do something like for frame in f: frame.get_data() to iterate over this set?

[pedrombmachado]

Hi @dorodnic and @tRosenflanz , Tried the following source code

import pyrealsense2 as rs import numpy as np

def initialize_camera():

start the frames pipe

p = rs.pipeline()
conf = rs.config()
conf.enable_stream(rs.stream.accel)
conf.enable_stream(rs.stream.gyro)
prof = p.start(conf)
return p

p = initialize_camera() f = p.wait_for_frames() for frame in f: print(frame.get_data())

and got

<pyrealsense2.pyrealsense2.BufData object at 0x7f6eb7120bc8> <pyrealsense2.pyrealsense2.BufData object at 0x7f6eb7120b90>

Any ideas on how to get the data from memory?

[tRosenflanz]

@pedrombmachado np.asanyarray should get the data

for frame in f: 
   print(np.asanyarray(frame.get_data()))

should have worked but produces [] as outputs.

After some digging this worked though:

for frame in f:
    print(frame.as_motion_frame().get_motion_data())

x: -0.0791966, y: -0.0623321, z: -9.29878
x: -0.000159076, y: -0.00338405, z: 2.59693e-06

One thing I dislike about this approach is that the API differs drastically from the one used for color,depth, and infrared channels but I guess that can be standardized by looping over all frames and calling is_X_frame() method on each to figure out what type they are.

I am gonna keep this open but you can close it if you don't think any changes to python wrapper are needed

[pedrombmachado]

Hi @tRosenflanz , Many thanks for your suggestion, I wrote the following script that can perhaps be made available to other users.

import pyrealsense2 as rs
import numpy as np

def initialize_camera():
    # start the frames pipe
    p = rs.pipeline()
    conf = rs.config()
    conf.enable_stream(rs.stream.accel)
    conf.enable_stream(rs.stream.gyro)
    prof = p.start(conf)
    return p

def gyro_data(gyro):
    return np.asarray([gyro.x, gyro.y, gyro.z])

def accel_data(accel):
    return np.asarray([accel.x, accel.y, accel.z])

p = initialize_camera()
try:
    while True:
        f = p.wait_for_frames()
        accel = accel_data(f[0].as_motion_frame().get_motion_data())
        gyro = gyro_data(f[1].as_motion_frame().get_motion_data())
        print("accelerometer: ", accel)
        print("gyro: ", gyro)

finally:
    p.stop()

Feel free to close it if you wish.

[RealSenseCustomerSupport]

---

HI tRosenflanz, pedrombmachado,

Thanks for the updates and feedback. Glad to see a working solution here. To my understanding, querying and then taking different work off a frame is current approach and may get some updates later on. I am closing this one for now if nothing else. Please check back later.

Thanks!

[ajpernalete]

Hi, I have this error print, when I used you code sample:

AttributeError: 'pyrealsense2.pyrealsense2.frame' object has no attribute 'as_motion_frame'

> import pyrealsense2 as rs
> import numpy as np
> 
> 
> def initialize_camera():
>     # start the frames pipe
>     p = rs.pipeline()
>     conf = rs.config()
>     conf.enable_stream(rs.stream.accel)
>     conf.enable_stream(rs.stream.gyro)
>     prof = p.start(conf)
>     return p
> 
> 
> def gyro_data(gyro):
>     return np.asarray([gyro.x, gyro.y, gyro.z])
> 
> 
> def accel_data(accel):
>     return np.asarray([accel.x, accel.y, accel.z])
> 
> p = initialize_camera()
> try:
>     while True:
>         f = p.wait_for_frames()
>         accel = accel_data(f[0].as_motion_frame().get_motion_data())
>         gyro = gyro_data(f[1].as_motion_frame().get_motion_data())
>         print("accelerometer: ", accel)
>         print("gyro: ", gyro)
> 
> finally:
>     p.stop()

[pedrombmachado]

Hi @ajpernalete try to update the pyrealsense2 library

$ pip install pyrealsense2 -U

Tested and it worked with the current version of realsense2

$ python test.py 

accelerometer: [ 0.52955908 -9.86548996 -0.18632634] gyro: [-0.00174533 0. 0. ] accelerometer: [ 0.52955908 -9.86548996 -0.18632634] gyro: [-0.00349066 0. 0. ] accelerometer: [ 0.52955908 -9.86548996 -0.18632634] gyro: [-0.00349066 0.00174533 -0.00174533] accelerometer: [ 0.52955908 -9.82626343 -0.16671304] gyro: [-0.00349066 0.00174533 -0.00174533] accelerometer: [ 0.52955908 -9.82626343 -0.16671304] gyro: [0. 0. 0.] accelerometer: [ 0.52955908 -9.82626343 -0.16671304] gyro: [-0.00174533 0.00523599 0. ] accelerometer: [ 0.52955908 -9.82626343 -0.16671304] gyro: [0. 0.00349066 0. ] accelerometer: [ 0.52955908 -9.84587669 -0.16671304]

[ajpernalete]

Hi @pedrombmachado , yes, I yesterday updated Pyrealsense and fix that, thanks anyway. Regards.

[Praneet1997]

I am not able to get accel and gyro working with depth or infra fields enabled. gyro and accel stream is working and infra images are working separately but not together. Any solution?

[malith1992]

Hello @pedrombmachado , Why is this gyro = gyro_data(f[1].as_motion_frame().get_motion_data()) don't give angle in degrees. Can you please tell me how to convert these gyro values to angles in degrees ?

Thanks

[magnusja]

I am not able to get accel and gyro working with depth or infra fields enabled. gyro and accel stream is working and infra images are working separately but not together. Any solution?

@Praneet1997 DO you have a solution for this? Stuck with the same problem ..

[pedrombmachado]

Sorry for my late reply. Guys the NIR are not aligned with RGB therefore is suggest a bed-time reading. @malith1992 you have to refer to the datasheet page 72. Also, @Praneet1997 you have to refer to the same datasheet page 62.

[Ahmad-M-Al-Khateeb]

I am not able to get accel and gyro working with depth or infra fields enabled. gyro and accel stream is working and infra images are working separately but not together. Any solution?

Sorry for the late reply, but I found that this is likely due to the index used for f changing after enabling the depth or color streams. So, in my case, after enabling the depth and color stream it was possible to access the accelerometer readings by using: accel = accel_data(f[2].as_motion_frame().get_motion_data()) rather than accel = accel_data(f[0].as_motion_frame().get_motion_data())

[vstadnytskyi-FDA]

import pyrealsense2 as rs import numpy as np

def initialize_camera():

start the frames pipe

p = rs.pipeline()
conf = rs.config()
conf.enable_stream(rs.stream.accel)
conf.enable_stream(rs.stream.gyro)
prof = p.start(conf)
return p

def gyro_data(gyro): return np.asarray([gyro.x, gyro.y, gyro.z])

def accel_data(accel): return np.asarray([accel.x, accel.y, accel.z])

p = initialize_camera() try: while True: f = p.wait_for_frames() accel = accel_data(f[0].as_motion_frame().get_motion_data()) gyro = gyro_data(f[1].as_motion_frame().get_motion_data()) print("accelerometer: ", accel) print("gyro: ", gyro)

finally: p.stop()

I am using this code but I am getting an error. I was able to use IMUs before. but something has happened and I cannot access IMUs anymore.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Input In [1], in <module>
     23 try:
     24     while True:
---> 25         f = p.wait_for_frames()
     26         accel = accel_data(f[0].as_motion_frame().get_motion_data())
     27         gyro = gyro_data(f[1].as_motion_frame().get_motion_data())

RuntimeError: Frame didn't arrive within 5000

It seems I am having issue discussed here https://support.intelrealsense.com/hc/en-us/community/posts/1500000585221-Cannot-getting-motion-data-from-L515 However, there is no solution to it. The conversation just stopped.