NickM-27
commented
2 years ago When you say recording stop, what does this mean?
What symptoms are you seeing that show recordings are not working?
Closed kspearrin closed 1 year ago
NickM-27
commented
2 years ago When you say recording stop, what does this mean?
What symptoms are you seeing that show recordings are not working?
kspearrin
commented
2 years ago When you say recording stop, what does this mean?
What symptoms are you seeing that show recordings are not working?
When I try to view events for that camera, I get errors like:
I assume this is because there is no recordings file for that timeframe.
I then look in the recordings directory and observe that no files are being created anymore for that camera. When I reboot Frigate, recordings files start showing up again every 10 seconds.
NickM-27
commented
2 years ago Do you see new recordings files in the container's /tmp/cache for that camera?
kspearrin
commented
2 years ago Do you see new recordings files in the container's /tmp/cache for that camera?
I observe that /tmp/cache has files being created and removed while the camera is operating normally. Whenever I restart the camera and the problem described above starts, the /tmp/cache directory has the last residual file that was created before the camera restarted and no new files are being created or deleted for that camera any longer. When I reboot Frigate, that residual file gets cleaned up and new ones start being created again, as expected.
hawkeye217
commented
2 years ago I notice the same thing with two of my cameras as well. Over the past few days I have been changing settings on my router and rebooting it. Frigate never records anything for those two particular cameras after the router reboots unless I restart Frigate as well.
Detect streams are still working - it's just recording that I have a problem with.
I may dig more into this later this week if there's no immediate solution.
s-cuba
commented
2 years ago Im having the same issues.
Im using 8 reolinks via rmtp streams and other than this and timestamps not working everything seems fine.
Whenever a camera leave the network, then rejoins frigate will not pick the stream back up.
Log [2022-04-27 14:21:43] watchdog.reo_2 INFO : No frames received from reo_2 in 20 seconds. Exiting ffmpeg... [2022-04-27 14:21:43] watchdog.reo_2 INFO : Waiting for ffmpeg to exit gracefully... [2022-04-27 14:22:13] watchdog.reo_2 INFO : FFmpeg didnt exit. Force killing... [2022-04-27 14:22:13] frigate.video ERROR : reo_2: Unable to read frames from ffmpeg process. [2022-04-27 14:22:13] frigate.video ERROR : reo_2: ffmpeg process is not running. exiting capture thread... [2022-04-27 14:22:23] watchdog.reo_2 ERROR : Ffmpeg process crashed unexpectedly for reo_2. [2022-04-27 14:22:23] watchdog.reo_2 ERROR : The following ffmpeg logs include the last 100 lines prior to exit.
Nothing is being recorded after it leaves. In the camera recodings tab it shows no times past the disconnect. It will for some reason still catch events, never record, just save a snapshot. When trying to view recording of said snapshot, the ui says network failure or incorrect format. Logs show
[2022-04-27 14:24:47] frigate.http ERROR : No recordings found for the requested time range
s-cuba
commented
2 years ago Well I have seemed to solve it, not sure what did it i changed some things in config, removed global args for ffmpeg as they were the same as default.
Heres my current config
#####
mqtt:
host: mosquitto_frigate
topic_prefix: xxx
client_id: xxx
user: xxx
password: xxx
####
ffmpeg:
input_args:
- -avoid_negative_ts
- make_zero
- -fflags
- nobuffer
- -flags
- low_delay
- -strict
- experimental
- -fflags
- +genpts+discardcorrupt
- -rw_timeout
- '5000000'
- -use_wallclock_as_timestamps
- "1"
- -f
- live_flv
###
detect:
width: 640
height: 480
fps: 5
enabled: True
###
objects:
track:
- person
- car
- bicycle
- bus
- cat
- dog
###
record:
enabled: True
retain:
days: 14
mode: all
events:
retain:
default: 14
mode: active_objects
###
snapshots:
enabled: True
retain:
default: 14
###
timestamp_style:
position: "tl"
format: "%m/%d/%Y %H:%M:%S"
####
cameras:
#1
reo_0:
ffmpeg:
inputs:
- path: rtmp://10.10.20.50/bcs/channel0_sub.bcs?channel=0&stream=0&user=xxx&password=xxx
roles:
- detect
- rtmp
- path: rtmp://10.10.20.50/bcs/channel0_main.bcs?channel=0&stream=0&user=xxx&password=xxx
roles:
- recordIm going to do some more testing to be sure its solved. All logs show the exact same info when a disconnect happens, but now never show these lines which show up in both of our error logs
[2022-04-18 21:23:59] watchdog.front_yard_ne INFO : No frames received from front_yard_ne in 20 seconds. Exiting ffmpeg... [2022-04-18 21:23:59] watchdog.front_yard_ne INFO : Waiting for ffmpeg to exit gracefully... [2022-04-18 21:24:29] watchdog.front_yard_ne INFO : FFmpeg didnt exit. Force killing...
So it seems as if frigate trying to kill ffmpeg causes the issue
stale[bot]
commented
2 years ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
kspearrin
commented
2 years ago Still an issue.
stale[bot]
commented
2 years ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
kspearrin
commented
2 years ago Still an issue
mikegleasonjr
commented
2 years ago When I was recording my RTSP streams manually with ffmpeg I had to add the option -stimeout 30000000 for ffmpeg to timeout whenever my camera/router was restarted. So I think it's the behavior of ffmpeg not Frigate. Otherwise ffmpeg would just hang without returning and without recording anything...
salleq
commented
2 years ago My workaround for this (having Reolink cameras that fail sometimes) is making Frigate be restarted on error. Basically I'm reading Home Assistan event log for: homeassistant.components.stream.stream.camera.CAMERANAME
and if that is found I send MQTT message to restart frigate.
I lose few seconds of reconrding whenever this happens but it's way better than to notice it many days later that a camera hasn't been recording (again).
kspearrin
commented
2 years ago My workaround for this (having Reolink cameras that fail sometimes) is making Frigate be restarted on error. Basically I'm reading Home Assistan event log for: homeassistant.components.stream.stream.camera.CAMERANAME
and if that is found I send MQTT message to restart frigate.
I lose few seconds of reconrding whenever this happens but it's way better than to notice it many days later that a camera hasn't been recording (again).
Are you doing this with a automation script of some sort or another way? Could you share more details?
salleq
commented
2 years ago Are you doing this with a automation script of some sort or another way? Could you share more details?
I do it with Node-Red. I believe it's doable with HA automations as well but I can't immediately say how to do that. I linked my flow here, it's not too complicated: https://pastebin.com/rEwcUvez
phedders
commented
2 years ago It would be really nice if Frigate could restart streams after they fail....
blakeblackshear
commented
2 years ago Frigate does restart streams when they fail. Pretty sure this is happening because the default input args were modified to remove the -timeout parameter.
phedders
commented
2 years ago Thankyou! I have input_args set - and wasnt aware of the defaults being overwritten... I'll poke it a bit without setting to see what I am missing.
kspearrin
commented
2 years ago @phedders Please let me know what you find. I was unable to get my streams working with the timeout input arg.
phedders
commented
2 years ago -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1
Was missing from the default options - so I have added that to my input_args and will see if it is more stable. Cheers
bagobones
commented
2 years ago Once or twice now I have found ALL of my camera feeds unresponsive old snapshot and no feed when opening the camera in the frigate web UI only two of my 6 cameras have custom input lines and the two that do have custom input still have the timeout.
In one of these cases I was able to restart frigate via the web UI restart, the other time I had to restart the docker..
I will try to capture logs next time but the WebUI it self still loaded. It was just the camera feeds being dead and the restart option did nothing in the one case.
NickM-27
commented
1 year ago Fixed in 0.12
Describe the problem you are having
This seems related to https://github.com/blakeblackshear/frigate/issues/1713
I have 10 Amcrest/Dahua turret cameras running on Frigate (see config below).
If a camera is restarted, Frigate will recover the event and live stream, however recordings stop and never recover until I restart Frigate.
I can simulate this by just manually restarting the camera.
Version
0.10.1-83481af
Frigate config file
Relevant log output
FFprobe output from your camera
Frigate stats
No response
Operating system
Debian
Install method
Docker Compose
Coral version
M.2
Network connection
Wired
Camera make and model
Amcrest IP8M-T2499EB-28MM
Any other information that may be helpful
Same issue seems to happen with all 10 of my cameras.