search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.46k stars 1.79k forks source link

[Support]: Frigate losing connection to camera's (No frames have been received error) #5650

Closed arjancool closed 1 year ago

arjancool commented 1 year ago

Describe the problem you are having

During a restart of the docker containr or after a while running the beta (8) version, Frigate loses connection to the camera's. The error i receive is : No frames have been received, check error logs.

docker error message : docker exec -it frigate-beta nvidia-smi Failed to initialize NVML: Unknown Error

if the error is received during the initial start this command is sufficient to get the camera's running : docker exec -it frigate-beta ldconfig

This error message is shown in the logs: ERROR : [h264_cuvid @ 0x55aa21307b80] Cannot load libnvcuvid.so.1 ERROR : [h264_cuvid @ 0x55aa21307b80] Failed loading nvcuvid.

after that : 

docker exec -it frigate-beta nvidia-smi

Mon Mar  6 08:45:45 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.161.03   Driver Version: 470.161.03   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 N/A |                  N/A |
| 31%   43C    P0    N/A /  N/A |    206MiB /  1999MiB |     N/A      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

if the error returns after running for a while, the ldconfig command is not working. this error is shown in the logs: ERROR : [AVHWDeviceContext @ 0x55dd382e2ac0] cu->cuInit(0) failed -> CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected

Version

0.12.0-27A31E7

Frigate config file

# yaml-language-server: $schema=http://10.0.0.60:5001/api/config/schema
mqtt:
  host: 10.0.0.12
  port: 1883
  topic_prefix: frigate-beta

database:
  path: /media/frigate/database/frigate.db

logger:
  default: info
  logs:
    frigate.event: debug

# Include all cameras by default in Birdseye view
birdseye:
  enabled: True
  mode: continuous

ffmpeg:
  hwaccel_args: -c:v h264_cuvid
    #ffmpeg:
    #hwaccel_args: preset-nvidia-h264

#detectors:
#  cpu1:
#    type: cpu
#    num_threads: 3
#  cpu2:
#    type: cpu
#    num_threads: 3

detectors:
  ov:
    type: openvino
    device: AUTO
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

objects:
  track:
    - person
    - car
    - bicycle
    - motorcycle
    - bus
    - boat
    - bird
    - cat
    - dog
    - umbrella

motion:
  threshold: 25
  contour_area: 30
  delta_alpha: 0.2
  frame_alpha: 0.2
  frame_height: 50
  mask: 0,900,1080,900,1080,1920,0,1920
  improve_contrast: True

record:
  enabled: True
  retain:
    days: 7
    mode: all
  events:
    retain:
      default: 28
      mode: motion

snapshots:
  enabled: true
  clean_copy: True
  timestamp: true
  bounding_box: False
  crop: False
  height: 175
  required_zones: []
  retain:
    default: 10
    objects:
      person: 15

timestamp_style:
  position: "tl"
  format: "%d-%m-%Y %H:%M:%S"
  color:
    red: 255
    green: 255
    blue: 255
  thickness: 2

go2rtc:
  streams:
    tuin: 
      - http://10.0.0.82/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=<username>&password=<password>
     # - ffmpeg:reolink#audio=opus
    parkeerplaats:
      - rtsp://<username>:<password>@10.0.0.81:554/videoMain
    deurbel:
      - rtsp:/<username>:<password>@10.0.0.35:554/mpeg/media.amp

cameras:
  tuin:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/tuin
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
            - detect
    zones:
      Gehele_tuin:
        coordinates: 0,0,0,720,1280,720,1280,0

  deurbel:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/deurbel
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
            - detect
    zones:
      Gehele_voordeur:
        coordinates: 0,720,0,0,1280,0,1280,720

  parkeerplaats:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/parkeerplaats
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record
            - detect
    zones:
      Regenton_zone:
        coordinates: 747,720,903,144,1280,147,1280,720
      Parkeerplaats_zone:
        coordinates: 0,720,0,0,1280,0,1280,720

Relevant log output

2023-03-06 08:44:52.170399173  [2023-03-06 08:44:52] watchdog.deurbel               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-03-06 08:44:52.170862561  [2023-03-06 08:44:52] ffmpeg.deurbel.detect          ERROR   : [h264_cuvid @ 0x55561184a240] Cannot load libnvcuvid.so.1
2023-03-06 08:44:52.171126417  [2023-03-06 08:44:52] ffmpeg.deurbel.detect          ERROR   : [h264_cuvid @ 0x55561184a240] Failed loading nvcuvid.

FFprobe output from your camera

not sure if its needed but :

ffprobe version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from 'rtsp://<username>:<password>@10.0.0.81:554/videoMain':
  Metadata:
    title           : videoMain
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1280x720, 50 tbr, 90k tbn, 180k tbc
  Stream #0:1: Audio: pcm_mulaw, 8000 Hz, 1 channels, s16, 64 kb/s

Frigate stats

No response

Operating system

Debian

Install method

Docker CLI

Coral version

CPU (no coral)

Network connection

Wired

Camera make and model

Reolink, Doorbird, Foscam

Any other information that may be helpful

No response

NickM-27 commented 1 year ago

What is your docker CLI command?

https://github.com/blakeblackshear/frigate/issues/5166

arjancool commented 1 year ago

i'd used this :

docker run -d \ --name frigate-beta \ --restart=unless-stopped \ --mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000 \ --device /dev/bus/usb:/dev/bus/usb \ --device /dev/dri/renderD128 \ --shm-size=64m \ --gpus all \ -v /media/Shares/frigate-beta/:/media/frigate \ -v /media/docker/frigate-beta/config/config.yml:/config/config.yml \ -v /etc/localtime:/etc/localtime:ro \ -e FRIGATE_RTSP_PASSWORD='' \ -e LIBVA_DRIVER_NAME=i965 \ -e TZ=Europe/Amsterdam \ -p 5000:5000 \ -p 1935:1935 \ ghcr.io/blakeblackshear/frigate:0.12.0-beta8

NickM-27 commented 1 year ago

Okay I'd check the linked issue, this is due to missing dependency on the host

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Roachforlife commented 1 year ago

Did you ever figure this one out?

arjancool commented 1 year ago

Did you ever figure this one out?

No, even with the latest release I've got this issue. Did a workaround to restart the container every day with the Nvidia smi command right after that.

boyphongsakorn commented 1 year ago

um.... is no way to fix that?