search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.21k stars 1.76k forks source link

[Support]: incomplete events and log output errors #2905

Closed blacknell closed 2 years ago

blacknell commented 2 years ago

Describe the problem you are having

I've tried to keep the set up as simple as possible but I'm experiencing

I'd like to invest in a coral but a bit reluctant until I can prove to myself that it's working in principle

I'm happy to rerun whatever tests you want with debug flags etc.

Version

0.10.0-bfecee9

Frigate config file

database:
  path: /media/frigate/frigate.db

# logger:
#   default: info
#   logs:
#     frigate.event: debug

cameras:
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://guest:REDACTED@192.168.100.67:554/
          roles:
            - record
        - path: rtsp://guest:REDACTED@192.168.100.67:554//h264Preview_01_sub
          roles:
            - detect
            - rtmp
    detect:
      width: 1280
      height: 720
      fps: 5

rtmp:
  enabled: true

birdseye:
  enabled: false

live:
  height: 360
  quality: 16

mqtt:
  host: homeassistant.local
  topic_prefix: frigate
  client_id: frigate
  user: mqtt
  password: REDACTED

detectors:
  cpu1:
    type: cpu
    num_threads: 4

objects:
  track:
    - person
    - car
    - dog
    - cat

record:
  enabled: true
  events:
      retain:
        default: 2

snapshots:
  enabled: True
  timestamp: True
  bounding_box: True

Relevant log output

example 1.
frigate  | [2022-03-05 15:29:38] watchdog.front_door            ERROR   : Ffmpeg process crashed unexpectedly for front_door.
frigate  | [2022-03-05 15:29:38] watchdog.front_door            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | [2022-03-05 15:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
frigate  | [2022-03-05 15:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Failed to update header with correct duration.
frigate  | [2022-03-05 15:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Failed to update header with correct filesize.

example 2
frigate  | [2022-03-05 13:29:38] watchdog.front_door            ERROR   : Ffmpeg process crashed unexpectedly for front_door.
frigate  | [2022-03-05 13:29:38] watchdog.front_door            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | [2022-03-05 13:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
frigate  | [2022-03-05 13:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Non-monotonous DTS in output stream 0:1; previous: 5980574, current: 5980477; changing to 5980574. This may result in incorrect timestamps in the output file.
frigate  | [2022-03-05 13:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Non-monotonous DTS in output stream 0:1; previous: 5980574, current: 5980566; changing to 5980574. This may result in incorrect timestamps in the output file.
frigate  | [2022-03-05 13:29:38] ffmpeg.front_door.detect       ERROR   : [rtsp @ 0x400404f0] DTS 148183070211089 < 148183070218789 out of order
frigate  | [2022-03-05 13:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Non-monotonous DTS in output stream 0:0; previous: 5980574, current: 5980489; changing to 5980574. This may result in incorrect timestamps in the output file.
frigate  | [2022-03-05 13:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Non-monotonous DTS in output stream 0:1; previous: 6850791, current: 6850776; changing to 6850791. This may result in incorrect timestamps in the output file.
frigate  | [2022-03-05 13:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Failed to update header with correct duration.
frigate  | [2022-03-05 13:29:38] ffmpeg.front_door.detect       ERROR   : [flv @ 0x400920d0] Failed to update header with correct filesize.

FFprobe output from your camera

ffprobe  rtsp://guest:REDACTED@192.168.100.67:554//h264Preview_01_sub
ffprobe version 4.3.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-omx --enable-omx-rpi --enable-mmal --enable-v4l2_m2m --enable-neon --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100

[rtsp @ 0x4002da90] UDP timeout, retrying with TCP
Input #0, rtsp, from 'rtsp://guest:REDACTED@192.168.100.67:554//h264Preview_01_sub':
  Metadata:
    title           : Session streamed by "preview"
    comment         : h264Preview_01_sub
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: h264, yuv420p(progressive), 640x360, 25 tbr, 90k tbn, 180k tbc
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp

ffprobe rtsp://guest:REDACTED@192.168.100.67:554/
ffprobe version 4.3.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-omx --enable-omx-rpi --enable-mmal --enable-v4l2_m2m --enable-neon --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
[rtsp @ 0x4002da90] UDP timeout, retrying with TCP
Input #0, rtsp, from 'rtsp://guest:guests@192.168.100.67:554/':
  Metadata:
    title           : Session streamed by "preview"
    comment         : 
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: hevc, yuv420p(tv), 3840x2160, 25 fps, 25 tbr, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp

Frigate stats

{

   "detection_fps":0.5,
   "detectors":{

      "cpu1":{

         "detection_start":1646556264.363336,
         "inference_speed":2394.58,
         "pid":930
      }
   },
   "front_door":{

      "camera_fps":5.1,
      "capture_pid":940,
      "detection_fps":0.5,
      "pid":936,
      "process_fps":0.3,
      "skipped_fps":0.0
   },
   "service":{

      "storage":{

         "/dev/shm":{

            "free":61.0,
            "mount_type":"tmpfs",
            "total":67.1,
            "used":6.1
         },
         "/media/frigate/clips":{

            "free":542977.0,
            "mount_type":"fuse.grpcfuse",
            "total":994662.6,
            "used":451685.6
         },
         "/media/frigate/recordings":{

            "free":542977.0,
            "mount_type":"fuse.grpcfuse",
            "total":994662.6,
            "used":451685.6
         },
         "/tmp/cache":{

            "free":994.0,
            "mount_type":"tmpfs",
            "total":1000.0,
            "used":6.0
         }
      },
      "temperatures":{

      },
      "uptime":35557,
      "version":"0.10.0-bfecee9"
   }
}

Operating system

Other

Install method

Docker Compose

Coral version

CPU (no coral)

Network connection

Wired

Camera make and model

Reolink 842A

Any other information that may be helpful

I'm running in docker compose with mqtt on the LAN. I have not connected this to HA yet. I also can't get the frigate home screen webcam image to stream (it's always a still shot)

NickM-27 commented 2 years ago

If the live view is "still" and not updating then that's what frigate sees too and would explain why things are being missed because the camera stream isn't working. Also keep in mind frigate does not create events on motion, only with true positive detected objects.

I also see you're using a reolink camera which is known to have many issues. Please refer to the docs on the config to attempt to improve this https://docs.frigate.video/configuration/camera_specific/#reolink-410520-possibly-others

blacknell commented 2 years ago

Ok. Will try those settings again later (I did see this mentioned and tried something like this without success) and report back.

I should point out

I think something is related to my detect settings. I've told it (if I'e understood it right) to do the detection from the low res stream 640x360 and yet defined the resolution at 1280x720; and the recording at the hires stream which is 4k. Also, won't the default object sizes be a factor?

NickM-27 commented 2 years ago

Ok. Will try those settings again later (I did see this mentioned and tried something like this without success) and report back.

Okay good to know, at the points where the web ui steam isn't updating then the ffmpeg process for that camera has errored out and is restarting.

I should point out

  • the camera seems to be streaming well - if I put the same URL's into VNC they work fine.

To be clear VLC is a good tool and indicator but if it works in VLC isn't a 100% guarantee it should / will work with ffmpeg in frigate

  • The video recordings it is creating do seem to be genuine ie they have a detected motion and also the box surrounding them categorising them as a person or car

Right, odds are when the camera works it works but the stream seems to be restarting which causes it to miss some things.

I think something is related to my detect settings. I've told it (if I'e understood it right) to do the detection from the low res stream 640x360 and yet defined the resolution at 1280x720; and the recording at the hires stream which is 4k. Also, won't the default object sizes be a factor?

Okay, you defined the resolution as different than the stream, ffmpeg will resize it manually to the detect size you specify. That will use more CPU but shouldn't affect object detection negatively. I see the higher quality stream recording, that is good. What do you mean by default object sizes?

blacknell commented 2 years ago

What do you mean by default object sizes?

I thought I read in the notes something about defining the size of detectable objects relative to the image size, but now I can't find it.

NickM-27 commented 2 years ago

What do you mean by default object sizes?

I thought I read in the notes something about defining the size of detectable objects relative to the image size, but now I can't find it.

You may be referring to this section: https://docs.frigate.video/guides/camera_setup which talks about the fact that the portion of the camera image is resized to 320x320 before object detection is done. The size of that is not configurable but yes the camera resolution affects it although just for smaller objects. Detecting cars or people that aren't on the other side of the street (just as an example) should be no issue.

blacknell commented 2 years ago

Ok. I'm going to abandon the Reolink and buy a Dahua. In fact I already have, should be here later this week.

Regarding a Coral, the guidance points to any of the Coral products but can I run it all on the Dev Board Mini (on Mendel Linux) or must I use a co-processor type via USB or PCI?

blacknell commented 2 years ago

@NickM-27 - ok, so I played around with Reolink settings and none of them worked with my Reolink 842A camera. Very disappointed (in the camera). FYI, seems to be an error in that camera setup above in that there are missing arguments - should be at least hwaccel_args: [].

Would appreciate a tip above on the Coral board

NickM-27 commented 2 years ago

That's unfortunate, that new camera should be quite a bit better.

As far as the coral goes, I believe it ~CAN~ but run on the dev board mini but I have seen a couple issues posted from users of them not working well. I am not knowledgeable in that regard so I am not sure exactly how viable that would be.

blacknell commented 2 years ago

My Dahua camera arrived and things definitely improved. But the frame rates were definitely very poor and hence loads of events were being missed. Then it occurred to me the docker image I was running was wrong. I'm running it on my Mac with Apple Silicon and had specified the wrong architecture. When I took out the tag and just specified blakeblackshear/frigate:release-0.11.0-cbf26e0 things improved dramatically. Even detected at 4K was giving my 5fps and 12ms inference speed.

blakeblackshear commented 2 years ago

I would suggest running blakeblackshear/frigate:0.10.1-aarch64 instead. The 0.11.0 build is still experimental.