Websocket or HTTP Detector

[2649]

I'm looking for a method to run larger models, that I trained on my own. Therefore I have some Nvidia Jetson boards, that run TensorRT. Furthermore, this would be an easy way to add integrations of custom models.

Although this adds the latency to inference time, it creates the possibility to scale horizontally.

Is this something, that is welcomed for the frigate?

[NickM-27]

CodeProject.AI detector in 0.13 runs on http but it does have slower inference speed than running direct detectors, frigate runs detection in real time so inference times are important. @blakeblackshear can comment further.

[blakeblackshear]

It remains to be seen whether or not the latency will be acceptable when sent over the network. Frigate doesn't use AI object detection like most other NVR systems. It doesn't simply send a frame periodically when motion is detected. It expects the inference times to be fast enough to track multiple objects in separate parts of the frame in real time for every single frame. Each frame may have object detection run several times for different portions of the frame. When you have multiple cameras with activity, you can easily see dozens to near 100 inferences per second.

[blakeblackshear]

It may make more sense to use the larger model as a second pass on snapshots like face or license plate detection is done.

[2649]

It may make more sense to use the larger model as a second pass on snapshots like face or license plate detection is done.

Yeah, that sounds also great. You could lower the detection threshold and remove false positives afterwards. Is this something you see in frigate or should it be in a separate project, that uses the API?

[blakeblackshear]

For now, I would suggest doing it in a separate project and using mqtt and the API.