search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.2k stars 1.66k forks source link

API endpoint that returns raw image data #1903

Open jakowenko opened 2 years ago

jakowenko commented 2 years ago

Describe what you are trying to accomplish and why in non technical terms To support third party applications that use Frigate's API, it would be helpful to have an endpoint that returns the raw image data.

This will help to avoid the overhead and data loss during the converting and decoding.

Describe the solution you'd like Updating the existing api/events/<id>/snapshot.jpg and api/<camera>/latest.jpg endpoints to support a new raw query string parameter. Ideally existing query string paramters would still be supported when passing raw=1, such as the ability to get a raw cropped with api/events/<id>/snapshot.jpg?raw=1&crop=1

I personally only have used snapshot.jpg and latest.jpg, but it might be helpful to add to all of the .jpg Frigate API endpoints listed here.

Describe alternatives you've considered Currently my API requests the JPG images from Frigate's API and then decodes it for processing. This would avoid some steps on both Frigate's side and the consumer of the API.

blakeblackshear commented 2 years ago

I'm not sure this will be the right approach. Once the event ends, the snapshot is converted to jpg and written to disk. It won't save much if frigate has to then convert the jpg to raw again. latest.jpg and snapshot.jpg come from memory during the event, so I think it may be best to have new endpoints for snapshot.raw and latest.raw instead. snapshot.raw may return a 404 once the event has ended.

ozett commented 2 years ago

what about clear up the snapshot.jpg/best.jpg/latest.jpg endpoint? into a single endpoint for images? with parameters for best/snap/latest and cams/events/.. and nearly similar options but an all this images?

jakowenko commented 2 years ago

I'm not sure this will be the right approach. Once the event ends, the snapshot is converted to jpg and written to disk. It won't save much if frigate has to then convert the jpg to raw again. latest.jpg and snapshot.jpg come from memory during the event, so I think it may be best to have new endpoints for snapshot.raw and latest.raw instead. snapshot.raw may return a 404 once the event has ended.

A snapshot.raw and latest.raw endpoint would be perfect. Wasn't sure what happened behind the scenes with the current ones, but having new endpoints to request raw would be great.

Do you think it makes sense to support the thumbnail and best images as well?

blakeblackshear commented 2 years ago

best would be relatively easy since those are in memory too, but thumbnail probably doesn't make much sense. I may add a format parameter here too. Technically frigate keeps frames in raw yuv420 pixel format, so you could get that as well. I already have a bunch of helper functions for cropping and converting from yuv420 to rgb so you dont have to do the work converting the areas of the image to rgb that aren't used.