PhidgetSpatial Precision IMU stops data output after first run on Jetson Orin

[Srijal97]

Phidgets Device: PhidgetSpatial Precision 3/3/3

Host Machine: Nvidia Jetson Orin Ubuntu 20.04 ROS Noetic

Issue Device gives /imu/data_raw output only on the first run. Subsequent launches throw no error, but there is no output seen on /imu/data_raw. It just remains empty until I unplug and replug the device again.

Example command: roslaunch phidgets_accelerometer accelerometer.launch Output:

... logging to /home/rezoom/.ros/log/812a18e4-fe33-11ec-9846-90e8688410b9/roslaunch-rezoom-orin-mi-3030.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://192.168.1.49:33071/

SUMMARY
========

PARAMETERS
 * /PhidgetsAccelerometerNodelet/data_interval_ms: 4
 * /rosdistro: noetic
 * /rosversion: 1.15.14

NODES
  /
    PhidgetsAccelerometerNodelet (nodelet/nodelet)
    accel_manager (nodelet/nodelet)

auto-starting new master
process[master]: started with pid [3060]
ROS_MASTER_URI=http://192.168.1.49:11311

setting /run_id to 812a18e4-fe33-11ec-9846-90e8688410b9
process[rosout-1]: started with pid [3092]
started core service [/rosout]
process[accel_manager-2]: started with pid [3095]
process[PhidgetsAccelerometerNodelet-3]: started with pid [3099]
[ INFO] [1657225771.798243280]: Loading nodelet /PhidgetsAccelerometerNodelet of type phidgets_accelerometer/PhidgetsAccelerometerNodelet to manager accel_manager with the following remappings:
[ INFO] [1657225771.800231370]: waitForService: Service [/accel_manager/load_nodelet] has not been advertised, waiting...
[ INFO] [1657225771.809492621]: Initializing nodelet with 12 worker threads.
[ INFO] [1657225771.821039510]: waitForService: Service [/accel_manager/load_nodelet] is now available.
[ INFO] [1657225771.825006537]: Initializing Phidgets Accelerometer Nodelet
[ INFO] [1657225771.825286161]: Starting Phidgets Accelerometer
[ INFO] [1657225771.828864429]: Connecting to Phidgets Accelerometer serial -1, hub port 0 ...
[ INFO] [1657225772.829471014]: Connected

It shows Connected but fails to output anything from the device after the first run onward. I check this using rostopic echo /imu/data_raw.

Upon closing using Ctrl+C

^C[PhidgetsAccelerometerNodelet-3] killing on exit
[ INFO] [1657226925.981820342]: Unloading nodelet /PhidgetsAccelerometerNodelet from manager accel_manager
[accel_manager-2] killing on exit
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

I have 2 of these PhidgetSpatial Precision devices and this issue occurs with both of them. Also note that this issue did not occur previously when I was testing this out on a Jetson Nano (Ubuntu 18 + ROS Melodic) or a Ubuntu 18 PC with ROS Melodic. We just switched to the Jetson Orin this week, which only comes with Ubuntu 20 (ROS Noetic) and have been facing this issue since.

Another interesting thing to note is that when I install libphidget22 and run the HelloWorld Example provided here, each plug and unplug event notification is printed out 5 times:

rezoom@rezoom-orin-mi:~/Downloads/HelloWorld_Example$ ./HelloWorld 
Opening...
Phidget Simple Playground (plug and unplug managers)
Press Enter to end anytime...
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373163
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535
Hello Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535
Goodbye Device PhidgetSpatial Precision 3/3/3, Serial Number: 373535

Also, I tried running lsusb and usbreset,

rezoom@rezoom-orin-mi:~$ lsusb
...
Bus 001 Device 013: ID 06c2:008c Phidgets Inc. (formerly GLAB) 
...

rezoom@rezoom-orin-mi:~$ usbreset 06c2:008c
Resetting PhidgetSpatial Precision 3/3/3 ... ok

And it fixes the issue momentarily as I get output on /imu/data_raw once. It stops working again on subsequent runs until I run usbreset again.

Apologies for the long post. Please let me know if any further details are required.

[mintar]

I didn't have this issue with other PCs with Ubuntu 20.04 / noetic, so it seems to be specific to the Nvidia Orin. I don't have an Orin for testing, so I can't reproduce and debug this issue. Please leave a comment if you find a solution.

[Srijal97]

Yes, I believe it is specific to the Orin too. I have not found a fix yet, but I will definitely post it here if I find something. Right now I am just physically unplugging/plugging the Phidget devices after each run of my robot.