Open hermy65 opened 1 year ago
Not really an expert but one way which may work would be to temporarily lower the confidence needed in frigate and this app and then hopefully you should see some random garden objects getting named as random birds. An other way if you have any video of birds, then you could feed this video as a camera stream using go2rtc. I've been using this second method to fine tune my pond heron detection.
Not really an expert but one way which may work would be to temporarily lower the confidence needed in frigate and this app and then hopefully you should see some random garden objects getting named as random birds. An other way if you have any video of birds, then you could feed this video as a camera stream using go2rtc. I've been using this second method to fine tune my pond heron detection.
@stevenbrown1862 Still grasping Frigate as a whole, how would i go about feeding a video into it using go2rtc?
I don't know how you have frigate set up currently. But adding something like the following to your streams section in the frigate config. You'll have to make sure that you've mounted the whole config folder, I think sometimes people only mount the config file.
streams: birdvideo: ffmpeg:/config/my video.mp4
You will then have this video looping at rtsp://frigate ip:8554/birdvideo and so you can point a frigate camera to it.
Sorry for any typos but tricky to type on the phone.
I don't know how you have frigate set up currently. But adding something like the following to your streams section in the frigate config. You'll have to make sure that you've mounted the whole config folder, I think sometimes people only mount the config file.
streams: birdvideo: ffmpeg:/config/my video.mp4
You will then have this video looping at rtsp://frigate ip:8554/birdvideo and so you can point a frigate camera to it.
Sorry for any typos but tricky to type on the phone.
definitely dont understand frigate well enough to make this work unfortunately. When i add in what you listed above with (i think)correct spacing frigate doesnt work. Looked at the documentation and didnt find anything similar to what you had up there either
If its not working then I think the most likely cause will be that you haven't mapped the config folder into your container. Sorry I've no idea about unraid so I've no idea how that would work. This link below is the documentation for streaming an mp4 file.
@stevenbrown1862 ok i finally got a video of birds streaming over rtsp and into a new camera in frigate. Now though, its not detecting them as birds at all, more so detecting them as people or not at all?? In the config, birdtest is the youtube video im streaming to frigate
Below is my frigate config, maybe i have something set incorrectly there? Appreciate the insight!!!
mqtt:
host: 192.168.0.134
port: 1883
topic_prefix: frigate
user: user
password: pass
stats_interval: 60
detectors:
tensorrt:
type: tensorrt
device: 0
model:
path: /trt-models/yolov7-tiny-416.trt
input_tensor: nchw
input_pixel_format: rgb
width: 416
height: 416
ffmpeg:
global_args: -hide_banner -loglevel warning
hwaccel_args: preset-nvidia-h264
input_args: preset-rtsp-generic
output_args:
# Optional: output args for detect streams (default: shown below)
detect: -threads 2 -f rawvideo -pix_fmt yuv420p
# Optional: output args for record streams (default: shown below)
record: preset-record-generic
detect:
width: 1920
height: 1080
enabled: true
objects:
track:
- bird
- dog
- person
filters:
bird:
threshold: 0.4
min_area: 5000
snapshots:
enabled: true
cameras:
birdcam:
record:
enabled: True
events:
pre_capture: 5
post_capture: 5
objects:
- bird
ffmpeg:
hwaccel_args: preset-nvidia-h264
inputs:
- path: rtsp://192.168.0.199:8554/cam
roles:
- detect
- record
birdtest:
record:
enabled: True
events:
pre_capture: 5
post_capture: 5
objects:
- bird
ffmpeg:
hwaccel_args: preset-nvidia-h264
inputs:
- path: rtsp://192.168.0.115:8554/live
roles:
- detect
- record
live:
stream_name: birdcam
mqtt:
enabled: True
bounding_box: False #this will get rid of the box around the bird. We already know it is a bird. Sheesh.
timestamp: False #this will get rid of the time stamp in the image.
quality: 95 #default quality is 70, which will get you lots of compression artifacts
Not sure on this one. I know frigate can be a bit random because of the coco model but not naming those pictures as birds is surprising. I'd suggest posting this on the frigate issues page to see if someone has any ideas.
@stevenbrown1862 posted over there, they said that the model im using isnt good at detecting birds but didnt offer a suggestion as to which model to use. Do you by chance know of what coco or yolo (or whatever its called) model is better at detecting birds?
Also, it did finally detect a couple birds but WhosAtMyFeeder didnt do anything with them still.
I've no idea about detectors sorry. I'm using yolov8 which is working well for me on my igpu. My use case is very different from you in that I'm trying to catch a heron stealing my fish. The easiest thing to see if your current detector is sub optimal would be to try running the default cpu detector to see if that detects better. When you can consistently get bird events then you could look in the who's at my feeder container logs as it will show the detection process assessing the picture from frigate there if its working.
Also, it did finally detect a couple birds but WhosAtMyFeeder didnt do anything with them still.
Are you seeing anything in the WhosAtMyFeeder console logs?
e.g.
Category(index=68, score=0.05859375, display_name='Cardinalis cardinalis', category_name='/m/01tgcy')
Getting image for event: 1688232892.504319-kg5mzz
Here's the URL: http://192.168.1.94:5000/api/events/1688232892.504319-kg5mzz/snapshot.jpg
The score is a percentage (threshold configurable in the app config through classification.threshold
) but it may give an indication of whether things are working or just need a bit of tuning. I've got a discussion thread open here pondering just this question...
perhaps @mmcc-xx may have some input on the model portion or why whosatmyfeeder lists nothing even though it detected 2 birds?
@daernsinstantfortress nothing, this is all i see in the logs, unless there is a different location for logging i should look at.
192.168.0.115 - - [30/Jun/2023 08:44:33] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:33] "GET /static/css/styles.css HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:38] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:38] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 08:44:38] "GET /static/images/flutter.svg HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:39] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:39] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 08:44:39] "GET /static/images/flutter.svg HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 10:49:59] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 17:04:43] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 18:16:28] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 18:16:29] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 18:16:30] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 18:16:30] "GET /static/images/flutter.svg HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:09] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:11] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [01/Jul/2023 10:26:11] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:11] "GET /static/images/flutter.svg HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:13] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [01/Jul/2023 10:26:13] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:13] "GET /static/images/flutter.svg HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 11:23:34] "GET /static/css/styles.css HTTP/1.1" 304 -
Calling Main
Time: 2023-06-30 08:43:48.205
Python version
3.8.17 (default, Jun 13 2023, 16:09:51)
[GCC 10.2.1 20210110]
Version info.
sys.version_info(major=3, minor=8, micro=17, releaselevel='final', serial=0)
Starting threads for Flask and MQTT
Starting flask app
* Serving Flask app 'webui'
* Debug mode: off
Starting MQTT client. Connecting to: 192.168.0.134:1883
Hmm is there anything after that, does it actually connect to the mqtt broker? I've pasted my logs below and you'll see there is confirmation of connection to mqtt.
Calling Main Time: 2023-06-22 11:13:58.076 Python version 3.8.17 (default, Jun 13 2023, 16:09:51) [GCC 10.2.1 20210110] Version info. sys.version_info(major=3, minor=8, micro=17, releaselevel='final', serial=0) Starting threads for Flask and MQTT INFO: Created TensorFlow Lite XNNPACK delegate for CPU. Starting flask app
@stevenbrown1862 interesting, so maybe its not actually connecting to mqtt then. in your config is it just ip:port or does it have the mqtt:// prefix? Mine is just ip:port and ive tried both true and false for auth and thats as far as it goes.
@hermy65 This is the relevant part of my config. You'll need to fix the indentations, I'm a bit rubbish at this github business.
frigate: frigate_url: http://192.168.1.191:5000 mqtt_server: 192.168.1.191 mqtt_auth: true mqtt_username: mqttuser mqtt_password: mqttpass main_topic: frigate
@stevenbrown1862 ok now we are making progress, changed the config to just mqtt ip NOT ip:port and it connected and started logging stuff like yours did. However, it is now identifying squirrels as morning doves :P
I guess this is why we don't need to worry about computers taking over the world and enslaving humans for a few years yet. I guess your way forward here, without having a better model, is to adjust the confidence percentages until your false positives are acceptable.
@stevenbrown1862 hate to keep hammering you on this but i just got my first bird and it looks like it id'd it in the logs but nothing is showing up on the webpage :/ Any idea what im doing wrong here?
I think the 'score' your seeing is the confidence so in other words its only 21-28% confident in the ID, therefore, it won't give you an ID in the UI and Frigate unless you lower your accepted confidence considerably (I wouldn't recommend these sort of levels though). I don't know whether the issue is the quality of the camera, the position of the camera, or just plain bad luck with how the bird was posing for you but to be honest I would probably ID that as a cat. Feel free to carry on hammering...at some point I may even have a useful answer.
Hi all - sorry for being tuned out for a while there. What @stevenbrown1862 said is correct - the confidence is below the threshold for a species identification to be considered real. That value is in the config file in the config directory.
Now, as to why Frigate isn't doing a great job with identifying birds in the first place...
I'd suggest trying a CPU based detector and the default built in model to see if you get any better results. You can back off on the fps in the camera section so it isn't hammering your CPU so hard. I also specify the resolution of my camera in the camera/detect section but I'm not sure how important that is.
Here's my camera definition in case it is helpful at all:
birdcam:
ffmpeg:
input_args: preset-rtmp-generic
inputs:
- path: rtmp://192.168.1.114/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=passwd
roles:
- detect
- record
output_args:
record: preset-record-generic-audio-copy
hwaccel_args: preset-vaapi
live:
stream_name: birdcam
detect:
enabled: True
width: 2560
height: 1920
fps: 6
record:
enabled: true
snapshots:
enabled: true
timestamp: False
bounding_box: False
mqtt:
bounding_box: False
timestamp: False
quality: 95
objects:
track:
- bird
- animal
filters:
animal:
threshold: 0.75
min_area: 5000
bird:
threshold: 0.6
min_area: 5000
I also specify the resolution of my camera in the camera/detect section but I'm not sure how important that is.
if you don't it will resize the stream to 1280x720
New to all of these software packages. I installed both Frigate and WhosAtMyFeeder on my unraid box. Frigate appears to be detecting what is needed, only had one detection of a bird and it was actually a squirrel. The issue im potentially running into is, how do i know if this is actually working. When Frigate classified the squirrel as a bird, it never showed up in WhosAtMyFeeder even though it was incorrect, i also did not see anything in the logs for WhosAtMyFeeder to suggest it decided against the squirrel.
So, how do i know this is working?
Below is my frigate config