Oak d poe with ROS

[antoineTK]

Hi guys!

I'm not sure this is the right place to be asking, but what is the best way to interface a Oak d poe camera with ROS nowadays? Would I use depthai-ros?

Thanks a lot

[Luxonis-Brandon]

@saching13 is best to advise. I think so, but I'm not sure if there are any specifics that need to be taken into account for PoE.

[saching13]

Yes. you can use depthai-ros. For ex: roslaunch depthai_examples stereo_node.launch monoResolution:=400p camera_model:=OAK-D-POE from here

[antoineTK]

sounds good! thank you guys

I have another question, regarding the poe injector. I see it needs to be 802.3af compliant, what about the speed? Would a 100MBps injector suffice?

[themarpe]

@antoineTK 1Gbps is highly recommended, as even at those speeds its much slower than USB models at USB3 Gen1 speeds

[antoineTK]

Got it thank you! I guess the 4k video will take most of that bandwidth, could you recommend a poe injector?

[themarpe]

@antoineTK More info on powering PoE devices in general here: https://docs.luxonis.com/projects/hardware/en/latest/pages/guides/powering_poe_devices.html

And WRT to injector we usually recommend: https://www.amazon.com/gp/product/B07JCB5XWF/

[antoineTK]

oh i missed that, great info on poe devices!

@saching13 to save on bandwidth can RGB image stream be disabled altogether? I see in the launch file I can change the resolution. We would only work with depth at the moment

[saching13]

I will modify the example to make that an argument.

[Serafadam]

Closing due to inactivity, if you think this should remain open, please tag me in a comment here.

[RhythmDJay]

@Serafadam Im currently trying to run depthai_examples/rgb_stereo_node.launch using an OAK-D PoE and getting the following error when launching. Is this due to using the PoE camera?

terminate called after throwing an instance of 'nanorpc::core::exception::logic'
  what():  Unexpected Encoding Type
================================================================================
REQUIRED process [rgb_stereo_publisher-3] has died!
process has died [pid 16705, exit code -6, cmd /mnt/wd/aggressionDetection/devel/lib/depthai_examples/rgb_stereo_node __name:=rgb_stereo_publisher __log:=/home/duane/.ros/log/0ad13794-768e-11ed-bcbf-af4e0674c733/rgb_stereo_publisher-3.log].
log file: /home/duane/.ros/log/0ad13794-768e-11ed-bcbf-af4e0674c733/rgb_stereo_publisher-3*.log
Initiating shutdown!
================================================================================

The log file also gives this as the only error straight after spin.

Any ideas?

[Serafadam]

Hi @RhythmDJay, can you check if you still have those errors when running stereo_intertial_node.launch? if yes, please run export DEPTHAI_DEBUG=1 and run it again to get some more info from the logs.

[RhythmDJay]

Hi @Serafadam, I can run stereo_inertial_node.launch I just get this error

Listing available devices...
Device Mx ID: 14442C10915F3FD700
[ERROR] [1670539039.680425422]: PluginlibFactory: The plugin for class 'rviz_imu_plugin/Imu' failed to load.  Error: According to the loaded plugin descriptions the class rviz_imu_plugin/Imu with base class type rviz::Display does not exist. Declared types are  rviz/AccelStamped rviz/Axes rviz/Camera rviz/DepthCloud rviz/Effort rviz/FluidPressure rviz/Grid rviz/GridCells rviz/Illuminance rviz/Image rviz/InteractiveMarkers rviz/LaserScan rviz/Map rviz/Marker rviz/MarkerArray rviz/Odometry rviz/Path rviz/PointCloud rviz/PointCloud2 rviz/PointStamped rviz/Polygon rviz/Pose rviz/PoseArray rviz/PoseWithCovariance rviz/Range rviz/RelativeHumidity rviz/RobotModel rviz/TF rviz/Temperature rviz/TwistStamped rviz/WrenchStamped rviz_plugin_tutorials/Imu
[14442C10915F3FD700] [192.168.0.10] [5.293] [SpatialDetectionNetwork(8)] [warning] Network compiled for 6 shaves, maximum available 10, compiling for 5 shaves likely will yield in better performance
Device USB status: UNKNOWN

[Serafadam]

Hi, seems like you don't have rviz plugins installed: sudo apt install ros-noetic-rviz-imu-plugin

[1412kauti]

I have the same error....and I already have ros-noetic-rviz-imu-plugin installed , it didnt work !,

any other fixes ?

behold the logs:

started roslaunch server http://comet:37723/

SUMMARY
========

PARAMETERS
 * /rgb_stereo_publisher/LRchecktresh: 5
 * /rgb_stereo_publisher/camera_param_uri: package://depthai...
 * /rgb_stereo_publisher/confidence: 200
 * /rgb_stereo_publisher/extended: False
 * /rgb_stereo_publisher/lrcheck: True
 * /rgb_stereo_publisher/subpixel: False
 * /rgb_stereo_publisher/tf_prefix: oak
 * /robot_description: <?xml version="1....
 * /rosdistro: noetic
 * /rosversion: 1.15.15

NODES
  /
    oak_state_publisher (robot_state_publisher/robot_state_publisher)
    rgb_stereo_publisher (depthai_examples/rgb_stereo_node)

auto-starting new master
process[master]: started with pid [2116356]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 6d819cda-835e-11ed-a679-032a457d4f74
process[rosout-1]: started with pid [2116379]
started core service [/rosout]
process[oak_state_publisher-2]: started with pid [2116386]
process[rgb_stereo_publisher-3]: started with pid [2116387]
[2022-12-24 07:41:51.179] [info] DEPTHAI_DEBUG enabled, lowered DEPTHAI_LEVEL to 'debug'
[2022-12-24 07:41:51.179] [debug] Library information - version: 2.19.1, commit:  from , build: 2022-11-28 18:15:09 +0000
[2022-12-24 07:41:51.180] [debug] Initialize - finished
[2022-12-24 07:41:51.235] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.21.tar.xz' open: 1ms, archive read: 54ms
[2022-12-24 07:41:51.396] [debug] Device - OpenVINO version: 2022.1
[2022-12-24 07:41:51.396] [debug] Device - BoardConfig: {"emmc":null,"gpio":[],"logDevicePrints":true,"logPath":null,"logSizeMax":null,"logVerbosity":null,"network":{"mtu":0,"xlinkTcpNoDelay":true},"pcieInternalClock":null,"sysctl":[],"uart":[],"usb":{"flashBootedPid":63037,"flashBootedVid":999,"maxSpeed":4,"pid":63035,"vid":999},"usb3PhyInternalClock":null,"watchdogInitialDelayMs":null,"watchdogTimeoutMs":null} 
libnop:
0000: b9 0e b9 05 81 e7 03 81 3b f6 81 e7 03 81 3d f6 04 b9 02 00 01 ba 00 be be bb 00 bb 00 be be be
0020: be be be 01
[2022-12-24 07:41:51.462] [debug] Resources - Archive 'depthai-device-fwp-adbcc016c8bd5a5580a26d8b6250f77160203666.tar.xz' open: 1ms, archive read: 280ms
[2022-12-24 07:41:52.073] [debug] Searching for booted device: DeviceInfo(name=1.2, mxid=184430103116060F00, X_LINK_BOOTED, X_LINK_USB_VSC, X_LINK_MYRIAD_X, X_LINK_SUCCESS), name used as hint only
[184430103116060F00] [1.2] [1.153] [system] [warning] PRINT:LeonCss: BootloaderConfig.options1 checksum doesn't match. Is: 0x10304418 should be: 0xFCCBD2D0
GPIO boot mode 0x16, interface USBD
Setting aons(0..4) back to boot from flash (offset = 0)
====ENABLE WATCHDOG====1
initial keepalive, countdown: 10
PLL0: 700000 AUX_IO0: 24000 AUX_IO1: 24000 MCFG: 24000 MECFG: 24000
Board init ret 3
eeprom configuration version: 55AA0006
Reading VERSION 6 --- ->
eeprom configuration load from user area, status: 0
--> brdInit ...
brdInitAuxDevices: Error: SC = 27: io_initialize expander_cam_gpios_1 [OK]

spi_N25Q_init: Flash JEDEC ID: ff ff ff
Invalid Flash JEDEC ID... No NOR available
Could not register the spi device
Opening bus for IR driver: 1
Failed to probe IR driver LM3644
Opening bus for IR driver: 2
Failed to probe IR driver LM3644
Opening bus for IR driver: 3
Failed to probe IR driver LM3644
Closing EEPROm!
MyriaX board configuration
pll0 frequency: 700000, ref0 frequency: 24000
Is booted from flash by bootloader: 0
Networking not available...
=== Enumerating on socket: Cam_A / RGB / Center
Probe failed I2C1 0x1a reg 0016: expected 0214, read 0378
Probe failed I2C1 0x1a reg 0016: expected 0214, read 0378
  >> Registered camera A12N02A (imx378) as /dev/Camera_0
Adding socket 0: cam 2. Sen name: IMX378
=== Enumerating on socket: Cam_B / Left
  >> Registered camera TG161B (ov9282) as /dev/Camera_1
Adding socket 1: cam 5. Sen name: OV9282
=== Enumerating on socket: Cam_C / Right
  >> Registered camera TG161B (ov9282) as /dev/Camera_2
Adding socket 2: cam 5. Sen name: OV9282
Initializing XLink...
UsbPumpVscAppI_Event: 5 VSC2_EVENT_ATTACH
UsbPumpVscAppI_Event: 2 VSC2_EVENT_SUSPEND
UsbPumpVscAppI_Event: 3 VSC2_EVENT_RESUME
UsbPumpVscAppI_Event: 4 VSC2_EVENT_RESET
initial keepalive, countdown: 9
UsbPumpVscAppI_Event: 4 VSC2_EVENT_RESET
UsbPumpVscAppI_Event: 4 VSC2_EVENT_RESET
UsbPumpVscAppI_Event: 4 VSC2_EVENT_RESET
UsbPumpVscAppI_Event: 0 VSC2_EVENT_INTERFACE_UP
Done!
Usb connection speed: High - USB 2.0
Temperature: Driver registered.
Temperature: Initialized driver.
Temperature: Sensor opened: CSS.
Temperature: Sensor opened: MSS.
Temperature: Sensor opened: UPA.
Temperature: Sensor opened: DSS.
[2022-12-24 07:41:53.243] [debug] Schema dump: {"connections":[{"node1Id":4,"node1Output":"video","node1OutputGroup":"","node2Id":5,"node2Input":"in","node2InputGroup":""},{"node1Id":2,"node1Output":"depth","node1OutputGroup":"","node2Id":3,"node2Input":"in","node2InputGroup":""},{"node1Id":1,"node1Output":"out","node1OutputGroup":"","node2Id":2,"node2Input":"right","node2InputGroup":""},{"node1Id":0,"node1Output":"out","node1OutputGroup":"","node2Id":2,"node2Input":"left","node2InputGroup":""}],"globalProperties":{"calibData":null,"cameraTuningBlobSize":null,"cameraTuningBlobUri":"","leonCssFrequencyHz":700000000.0,"leonMssFrequencyHz":700000000.0,"pipelineName":null,"pipelineVersion":null,"xlinkChunkSize":-1},"nodes":[[0,{"id":0,"ioInfo":[[["","out"],{"blocking":false,"group":"","name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"MonoCamera","properties":[185,7,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,1,255,0,136,0,0,240,65,3,3]}],[1,{"id":1,"ioInfo":[[["","out"],{"blocking":false,"group":"","name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"MonoCamera","properties":[185,7,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,2,255,0,136,0,0,240,65,3,3]}],[2,{"id":2,"ioInfo":[[["","depth"],{"blocking":false,"group":"","name":"depth","queueSize":8,"type":0,"waitForMessage":false}],[["","rectifiedLeft"],{"blocking":false,"group":"","name":"rectifiedLeft","queueSize":8,"type":0,"waitForMessage":false}],[["","syncedLeft"],{"blocking":false,"group":"","name":"syncedLeft","queueSize":8,"type":0,"waitForMessage":false}],[["","syncedRight"],{"blocking":false,"group":"","name":"syncedRight","queueSize":8,"type":0,"waitForMessage":false}],[["","rectifiedRight"],{"blocking":false,"group":"","name":"rectifiedRight","queueSize":8,"type":0,"waitForMessage":false}],[["","outConfig"],{"blocking":false,"group":"","name":"outConfig","queueSize":8,"type":0,"waitForMessage":false}],[["","debugExtDispLrCheckIt2"],{"blocking":false,"group":"","name":"debugExtDispLrCheckIt2","queueSize":8,"type":0,"waitForMessage":false}],[["","debugDispCostDump"],{"blocking":false,"group":"","name":"debugDispCostDump","queueSize":8,"type":0,"waitForMessage":false}],[["","confidenceMap"],{"blocking":false,"group":"","name":"confidenceMap","queueSize":8,"type":0,"waitForMessage":false}],[["","debugDispLrCheckIt2"],{"blocking":false,"group":"","name":"debugDispLrCheckIt2","queueSize":8,"type":0,"waitForMessage":false}],[["","debugDispLrCheckIt1"],{"blocking":false,"group":"","name":"debugDispLrCheckIt1","queueSize":8,"type":0,"waitForMessage":false}],[["","debugExtDispLrCheckIt1"],{"blocking":false,"group":"","name":"debugExtDispLrCheckIt1","queueSize":8,"type":0,"waitForMessage":false}],[["","disparity"],{"blocking":false,"group":"","name":"disparity","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","name":"inputConfig","queueSize":4,"type":3,"waitForMessage":false}],[["","left"],{"blocking":false,"group":"","name":"left","queueSize":8,"type":3,"waitForMessage":true}],[["","right"],{"blocking":false,"group":"","name":"right","queueSize":8,"type":3,"waitForMessage":true}]],"name":"StereoDepth","properties":[185,16,185,5,185,9,0,2,136,0,0,122,68,2,0,0,5,3,0,185,7,5,0,185,5,0,2,136,0,0,0,63,0,1,185,4,0,3,136,205,204,204,62,0,185,2,0,134,255,255,0,0,185,2,0,50,185,2,1,0,185,4,255,0,1,0,185,5,1,0,0,128,200,185,3,0,2,127,185,5,1,128,250,129,244,1,128,250,129,244,1,255,1,0,190,190,190,190,1,185,5,189,0,189,0,190,16,16,0,3,255,255,1,190]}],[3,{"id":3,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,5,100,101,112,116,104,0]}],[4,{"id":4,"ioInfo":[[["","video"],{"blocking":false,"group":"","name":"video","queueSize":8,"type":0,"waitForMessage":false}],[["","still"],{"blocking":false,"group":"","name":"still","queueSize":8,"type":0,"waitForMessage":false}],[["","isp"],{"blocking":false,"group":"","name":"isp","queueSize":8,"type":0,"waitForMessage":false}],[["","preview"],{"blocking":false,"group":"","name":"preview","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"ColorCamera","properties":[185,23,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,0,1,0,129,44,1,129,44,1,255,255,255,255,0,136,0,0,240,65,136,0,0,128,191,136,0,0,128,191,1,185,4,0,0,0,0,3,3,4,4,4]}],[5,{"id":5,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,5,118,105,100,101,111,0]}]]}
[2022-12-24 07:41:53.243] [debug] Asset map dump: {"map":{}}
[184430103116060F00] [1.2] [1.155] [system] [info] Memory Usage - DDR: 0.12 / 340.61 MiB, CMX: 2.05 / 2.50 MiB, LeonOS Heap: 7.24 / 77.48 MiB, LeonRT Heap: 2.89 / 41.35 MiB
[184430103116060F00] [1.2] [1.155] [system] [info] Temperatures - Average: 31.16 °C, CSS: 32.49 °C, MSS 30.32 °C, UPA: 31.04 °C, DSS: 30.80 °C
[184430103116060F00] [1.2] [1.156] [system] [info] Cpu Usage - LeonOS 27.34%, LeonRT: 1.49%
[184430103116060F00] [1.2] [1.163] [system] [warning] PRINT:LeonCss: I: [Timesync] [   1745680] [XLin] startSync:130     Timesync | Callback not set
[2022-12-24 07:41:53.392] [debug] Device about to be closed...
[2022-12-24 07:41:53.496] [debug] DataOutputQueue (video) closed
[2022-12-24 07:41:53.496] [debug] Timesync thread exception caught: Couldn't read data from stream: '__timesync' (X_LINK_ERROR)
[2022-12-24 07:41:53.496] [debug] Log thread exception caught: Couldn't read data from stream: '__log' (X_LINK_ERROR)
[2022-12-24 07:41:53.496] [debug] DataOutputQueue (depth) closed
[2022-12-24 07:41:53.986] [debug] Watchdog thread exception caught: Couldn't write data to stream: '__watchdog' (X_LINK_ERROR)
[2022-12-24 07:41:54.368] [debug] XLinkResetRemote of linkId: (0)
[2022-12-24 07:41:54.734] [debug] Device closed, 1342
terminate called after throwing an instance of 'nanorpc::core::exception::logic'
  what():  Unexpected Encoding Type
================================================================================REQUIRED process [rgb_stereo_publisher-3] has died!
process has died [pid 2116387, exit code -6, cmd /home/kaito/catkin_ws/devel/lib/depthai_examples/rgb_stereo_node __name:=rgb_stereo_publisher __log:=/home/kaito/.ros/log/6d819cda-835e-11ed-a679-032a457d4f74/rgb_stereo_publisher-3.log].
log file: /home/kaito/.ros/log/6d819cda-835e-11ed-a679-032a457d4f74/rgb_stereo_publisher-3*.log
Initiating shutdown!
================================================================================
[rgb_stereo_publisher-3] killing on exit
[oak_state_publisher-2] killing on exit
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

[Serafadam]

Hi @1412kauti do you still have those errors when using stereo_inertial_node launch?

[abegni]

@Serafadam Im currently trying to run depthai_examples/rgb_stereo_node.launch using an OAK-D PoE and getting the following error when launching. Is this due to using the PoE camera?

terminate called after throwing an instance of 'nanorpc::core::exception::logic'
  what():  Unexpected Encoding Type
================================================================================
REQUIRED process [rgb_stereo_publisher-3] has died!
process has died [pid 16705, exit code -6, cmd /mnt/wd/aggressionDetection/devel/lib/depthai_examples/rgb_stereo_node __name:=rgb_stereo_publisher __log:=/home/duane/.ros/log/0ad13794-768e-11ed-bcbf-af4e0674c733/rgb_stereo_publisher-3.log].
log file: /home/duane/.ros/log/0ad13794-768e-11ed-bcbf-af4e0674c733/rgb_stereo_publisher-3*.log
Initiating shutdown!
================================================================================

The log file also gives this as the only error straight after spin.

Any ideas?

Same error, any ideas?

[Serafadam]

Hi, could you post more logs according to the bug template? It would be also goo to create a new issue for that.