Deepstream integration to frigate

[LeDuySon]

Describe what you are trying to accomplish and why in non technical terms I want to be able to like some how integrate Deepstream with frigate, maybe integrate inside or outside (not sure it will work). The reason i want to integrate deepstream because its very optimized for video processing and also have many models that i need

Describe the solution you'd like The solution right now that im thinking of is:

• Deepstream will read the RTSP, process it and output another RTSP
• Then, frigate will read that output RTSP to show the results which include object detection and tracking or maybe counting on the screen (So frigate maybe work only as the frontend). Right now, im not sure how deepstream will communicate with frigate to send information like bounding boxes, id, violence detection to trigger event. And i also want to trigger the recording if frigate receive violence event (like the mode: motion in recording section).

Questions Im new in frigate so i want to ask a couple of questions:

• Do you think the solution that i mentioned above can be done?
• How can we do the communicate things?
• Is frigate support many detectors models run in the same time?
• Can we customize easily the recording trigger based on other events but not motion?
• Do you guys have a plan to support to send api to inference server (triton inference server) for processing the frame?

Notes I want to deep dive into it to answer the question myself but we dont have much time so i need to consider it will work for our case then we can have a deep look into frigate. Btw, thank you guys for a great application, really love it!

[NickM-27]

Do you think the solution that i mentioned above can be done?

difficult to say without doing more research on deepstream, and I'm not sure in general if that's the direction frigate is looking to go (question for Blake)

Is frigate support many detectors models run in the same time?

not currently, no

Do you guys have a plan to support to send api to inference server (triton inference server) for processing the frame?

CodeProjectAI via REST is already supported, TCP based communication is slower than native coral, OpenVINO, tensorrt, etc. though

https://github.com/blakeblackshear/frigate/blob/080d7a2d886386daf15eafd54e5fe49969e9592e/frigate/detectors/plugins/deepstack.py#L18-L26

[LeDuySon]

Thank you for really fast response @NickM-27. Its interesting that is support REST API (I dont see in the documetation, maybe i missed that), so basically i can send a frame to triton inference server and run inference with many models in the inference server. Do you guys have plan for support gRPC?

[NickM-27]

(I dont see in the documetation, maybe i missed that),

it is currently in the dev branch / 0.13 (currently in beta)

[LeDuySon]

Thanks. And what about the communication, is there a way that i can send event from another application to frigate?

[NickM-27]

Currently available in dev / 0.13 beta https://deploy-preview-6262--frigate-docs.netlify.app/integrations/api#post-apieventscamera_namelabelcreate

[LeDuySon]

Great, thank you. And also can we customize easily the recording trigger based on other events but not motion? Like i want to send an event from an outside application and trigger the recording process

[NickM-27]

include_recordings is one of the body parameters in that request, shown in the docs that I sent. The default record config (except setting recordings to be enabled) should work

[LeDuySon]

Oh great, i will have a look at that. Thank you so much @NickM-27

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.