luxonis / depthai-ros

Official ROS Driver for DepthAI Sensors.
MIT License
239 stars 173 forks source link

Feed recorded frames into XLinkIn Node #312

Closed ninoweg closed 10 months ago

ninoweg commented 1 year ago

I recorded color/left/right streams as mp4 as described in Recording and I would now like to feed them into my c++ pipeline (similar to stereo_intertial_publisher.cpp) instead of the physical camera.

In Replaying it states that that results in using XLinkIn, but how can I send recorded frames to XLinkIn nodes? I couldn't find any examples on how to do so.

Serafadam commented 1 year ago

Hi, we don't have it currently in depthai_examples, but you can check out depthai-core mobilenet example, or if you want to pass data from ROS topic, depthai_ros_driver.

ninoweg commented 1 year ago

Thank's for the help, I managed to get everything running inside a ROS node. I guess I overlooked depthai-core mobilenet example.
Regarding the recordings: It only seems to be possible to feed ColorCamera.video (or also MonoCamera.out) output directly into the VideoEncoder whereas when using ColorCamera.isp or ColorCamera.preview I get an error regarding the frame type: [VideoEncoder(8)] [warning] Arrived frame type (8) is not either NV12 or YUV400p (8-bit Gray). I solved this by using ImageManip and by converting frame type to NV12. Is it also possible to feed RGB frames directly into the encoder or do I need to use ImageManip?

ninoweg commented 1 year ago

Is there maybe also an example on how to create StereoDepth from recordings? I'm recorded both left and right MonoCamera and I'm feeding that into a StereoDepth node. I don't get any error message but the node doesn't output any depth either. I used the same setup as in depthai-core mobilenet example. My guess is that this might also be an format issue

Serafadam commented 1 year ago

Hi @ninoweg, I guess this is related to InstanceNum in message, you need to set it so that Stereo node knows which camera ID is related to which message, you can see the example in depthai_ros_driver

ninoweg commented 1 year ago

That was it, thanks a lot!

Serafadam commented 10 months ago

Closing due to inactivity, please reopen if there are still questions.