[Feature-Request] basic depthai simulator

[Aposhian]

Start with the why:

When writing production software against depthai, it is desirable to have something to test against with zero hardware requirements, in order to validate basic things like:

• does my software correctly attempt to connect to a depthai-enabled device?
• does my software respond to common depthai failure modes

Move to the what:

I think there are a few different levels of functionality:

1. basic connection simulation: accept connections, inject disconnects
2. inject uninteresting mock data based on pipeline output queues (similar to videotestsrc for gstreamer)
3. ultimate: connect to camera plugins for physics simulations like gazebo and output test data tied to the simulation.

However, the goal would never be to match feature parity to a real device. Most things should be faked and mocked. This sort of simulator is primarily to test software infrastructure, validate pipeline correctness, etc.

Move to the how:

I think the minimum I would need to get started on this is:

• good documentation of XLink protocol messages

and then features could be incrementally added.

[themarpe]

Hi @Aposhian

Thanks for the request.

Simulating the whole device functionality is unfortunately out of scope.

However, the goal would never be to match feature parity to a real device. Most things should be faked and mocked. This sort of simulator is primarily to test software infrastructure, validate pipeline correctness, etc.

Handling things in a mock manner would however be applicable. We are making some great strides towards this with RVC3 support, but not there yet.

I think the minimum I would need to get started on this is: good documentation of XLink protocol messages

XLink is open, but not well documented: https://github.com/luxonis/xlink

Check: develop_server & develop_server_testing_race_improvements branches for "server" parts of XLink.

---

I think that something along these lines should be doable to add, hopefully this year, in limited fashion.

[Aposhian]

Thanks! I'll take a look at those branches.