Closed Curid closed 2 years ago
This would improve the consistency between the day and night accuracy.
FWIW, if night = infrared, I'm not sure how true this is. I've found that things that reflect visible light often don't reflect infrared and vice versa. So I suspect converting to grayscale won't be a great substitute for having training data of both types. I'd expect it to worsen daytime results (because you're discarding information). I wouldn't even be surprised if it worsens IR results (because of similar-seeming but inconsistent training data). YMMV, actual experiments win over my intuition, etc.
By "black and white" I mean RGB frames with saturation of 0, not grayscale.
Why not use true grayscale then so there's less data to process?
I'm working on a custom model trained only on gray images. https://gitlab.com/Curid/TF-CCTV
Neat! I've been wanting an open CCTV model for a while, so I'm excited to see that. I'll be watching it and might contact you to contribute to it.
I suspect converting to grayscale won't be a great substitute
It's a fast filter that can be applied to both day and night images. I considered a fancier filter with blur etc, but then it probably wouldn't work at day time and would need to be toggled.
I'd expect it to worsen daytime results
This is correct, but I think the tradeoff is worth it, if it improves the consistency.
I wouldn't even be surprised if it worsens IR results (because of similar-seeming but inconsistent training data).
We'll see. The model trained only on gray images surprisingly still works on color images with almost the same accuracy as the gray ones.
Why not use true grayscale then so there's less data to process?
It would require changes to DOODS and Frigate to be compatible. And Transfer learning wouldn't work with the original model. Maybe the original model can be converted to grayscale but i don't know how to do it.
Neat! I've been wanting an open CCTV model for a while, so I'm excited to see that. I'll be watching it and might contact you to contribute to it.
That would be great! Are you planning to add object detection to Moonfire NVR?
Are you planning to add object detection to Moonfire NVR?
Yes. I'd be happy to talk with you about this more elsewhere (as my plans don't have much impact on this doods feature request).
I am actually in process of rewriting DOODS in python with support for Tensorflow 2 Object Detection API so it can use all those models (As well as the python code is much easier to work with) I can have a look at this after that is done. Would it be a global option or just an option as part of the detection? Would it be possible to add the greyscale conversion as part of the process of running the model?
Would it be a global option or just an option as part of the detection?
Preferably in the detector config next to numThreads
.
Would it be possible to add the greyscale conversion as part of the process of running the model?
Do you mean before sending to DOODS? That's what I'm currently doing.
Okay, have a look, it's completed. The request object now includes a preprocess option where you can pass grayscale
.
https://github.com/snowzach/doods2
Are you calling this from Home Assistant?
Are you calling this from Home Assistant?
It's called from OS-NVR using gRPC.
The frame is first scaled, masked, cropped and converted to grayscale if the model name starts with gray_
, before being sent to doods Current implementation.
Looks like doods2 doesn't support gRPC. I was hoping that it would be replaced by WebSockets anyway, but it doesn't seem to currently have the same functionality gRPC did. Draft
Ohhh! That's a cool looking project. Never seen it... Okay, I added a /detect
websocket endpoint. I am not too good at Python so I'm not sure if it's handling it asynchronously right. Have a look and let me know.
@scottlamb not sure if you've seen but https://github.com/snowzach/doods2 supports opening a websocket, passing it an RTSP URL and just listening for detections in realtime.
Since I've added support for grayscale pre-processing and detection websockets, I'm going to close the issue. Feel free to open more PRs in the new repo. Thanks for the suggestions.
This would improve the consistency between the day and night accuracy. By "black and white" I mean RGB frames with saturation of 0, not grayscale. The FFmpeg equivalent would be
ffmpeg -i input.png -vf hue=s=0 output.png
I'm working on a custom model trained only on gray images. https://gitlab.com/Curid/TF-CCTV