[BUG] Low FPS and unstable frame time/power draw with IMX462 and Yolov6 custom models

[atmccarthy]

Describe the bug On the IMX 283 branch with an IMX462 camera, Yolov6n custom models run at a very low frame rate when the ColorCamera node is set to certain FPS values, such as 15 FPS. The same behaviour is not observed with other cameras on the main branch, or with the stock/COCO yolov6n model on the imx283 branch. I have also noticed that the time between frames varies a lot, and that device's power draw is quite unstable. Also, if you set the FPS above 60, then the output becomes heavily distorted, and the device crashes a lot.

Minimal Reproducible Example I have observed this bug with an OAK-FFC 4p with the camera installed on socket D. I have not tried the other boards/sockets.

1. Train a custom yolov6n model using the chess sample dataset. I have attached such a model to this bug report.
2. Make sure you have the imx283 branch checked out and have the correct firmware and other dependencies for this branch installed via pip.
3. Modify the yolov6 example so that it uses the model from step 1 and set the camRGB FPS to 15
4. Run the pipeline, and observe the FPS

Actual FPS = 5 Expected FPS = 15

Setting the FPS to 20 gives a better actual frame rate of ~14FPS, but still less than the 20 asked for.

Pipeline Graph

model.zip

[Erol444]

Hi @atmccarthy , Are you reading frames fast enough on the host side? And what's the FPS of the model itself? Some useful docs about queues/pools and blocking/nonblocking behaviour can be found here: https://docs.luxonis.com/projects/api/en/latest/components/nodes/

[atmccarthy]

Hi @Erol444 ,

Are you reading frames fast enough on the host side? And what's the FPS of the model itself?

If i set a high frame rate on the camera node, e.g 60FPS, then the model is able to run at ~25 FPS. The problem is that when setting lower frame rates the actual FPS of the pipeline is much lower, e.g. setting the camera node to 15 FPS achieves a rate of 5 FPS. The same behaviour is not observed on the main branch with other cameras.

[themarpe]

@atmccarthy this could be due to NN not being constantly fed with new data - setting the NN input to non blocking and higher frame rate, will be more likley that there will always be a frame ready for consumption. Also, what is the queue size of the NN input? Try increasing that to "num threads", by default likely around 3 - and see if framerate improves with 15FPS on sensor

[atmccarthy]

@themarpe, the code i'm running is the example, which already has blocking set to false on its input. Also, please note that this behaviour is present with the IMX462 on the IMX283 branch only; the exact same code/pipeline works fine on the main branch.

[themarpe]

@atmccarthy do you mind modifying also:

detectionNetwork.input.setBlocking(False)
detectionNetwork.input.setQueueSize(3) # or try also with 2

It could be that IMX462 outputs frames in a more bursty manner, which can cause the above. If you don't mind giving above tweak a try and let me know if that works for you.

[atmccarthy]

Hi @themarpe,

The problem appears to be with my FFC-4P board, as I can only reproduce the issue on that board, and not my POE 1P board. Another interesting thing to note is that the camera only works with the CAM_D socket of my 4P board. When I connect the camera to the CAM_A socket, the device will either crash or produce a solid orange picture. Setting the queue size to 3 when using CAM_D didn't help.

You could try reproducing this on your FFC-4P board, and if you can't, then we can conclude that my 4P is faulty.