search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.21k stars 1.76k forks source link

[Support]: RTSP server not working #4986

Closed atv2016 closed 1 year ago

atv2016 commented 1 year ago

Describe the problem you are having

I am trying to get the birdseye rtsp view on my tv, but whenever i try to pull it (i use webrtc dash cast to put the rtsp stream on it) i see this in the frigate logs:

.934 WRN github.com/AlexxIT/go2rtc/cmd/rtsp/rtsp.go:202 > error=EOF VLC also bombs out.

In addition, my camera entities from frigate that i stream or record are still are as slow as 30seconds behind, which i don't understand. When i used generic camera entities i'm pretty sure it was close to realtime. Maybe it also has something to do with this error.

Version

SYSTEM 0.12.0-0DBF909

Frigate config file

(This is my old frigate.yml for the stable release)

mqtt:

  host: 192.168.50.148
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: mqtt
  password: test
  stats_interval: 300

record:
  expire_interval: 10

timestamp_style:
  format: "%m/%d/%Y %H:%M:%S"
  color:
    red: 255
    green: 255
    blue: 255
  thickness: 2
  effect: shadow

cameras:

  license_plate:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.15:554/user=admin&password=test&channel=5&stream=0.sdp?real_stream--rtp-caching=100
          roles:
            - detect
            - record
    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      track:
        - person
        - dog
        - cat
        - mouse
        - car
      filters:
        car:
          min_area: 15000
        person:
          min_area: 500
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
#      required_zones:
#        - front_driveway_1_close_in
#        - front_driveway_1_whole_area
      crop: True
      height: 500
      retain:
        default: 3
 #   zones:
 #     front_driveway_1_close_in:
 #       coordinates: 0,480,429,480,443,335,460,173,263,133,88,120,36,212,0,279
 #       objects:
 #         - car
 #         - person
 #         - dog
 #         - cat
 #         - bicycle
 #     front_driveway_1_whole_area:
 #       coordinates: 383,52,497,105,569,316,575,480,261,480,0,480,0,225,148,57
 #       objects:
 #         - person
 #         - cat
 #         - dog
 #         - bicycle
    motion:
      mask:
        - 1211,102,1682,101,1679,62,1211,64
    record:
      enabled: True
      retain:
        days: 0
      events:
        retain:
          default: 5
          mode: motion
#        required_zones:
#          - front_driveway_1_close_in
#          - front_driveway_1_whole_area
        pre_capture: 5
        post_capture: 15

  greenhouse_2:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.170:554/user=admin&password=test&channel=3&stream=0.sdp?real_stream--rtp-caching=100
          roles:
            - detect
            - record
    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      track:
        - person
        - dog
        - cat
        - mouse
      filters:
        car:
          min_area: 15000
        person:
          min_area: 500
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
#      required_zones:
#        - front_driveway_1_close_in
#        - front_driveway_1_whole_area
      crop: True
      height: 500
      retain:
        default: 3
 #   zones:
 #     front_driveway_1_close_in:
 #       coordinates: 0,480,429,480,443,335,460,173,263,133,88,120,36,212,0,279
 #       objects:
 #         - car
 #         - person
 #         - dog
 #         - cat
 #         - bicycle
 #     front_driveway_1_whole_area:
 #       coordinates: 383,52,497,105,569,316,575,480,261,480,0,480,0,225,148,57
 #       objects:
 #         - person
 #         - cat
 #         - dog
 #         - bicycle
    motion:
      mask:
        - 1211,102,1682,101,1679,62,1211,64
    record:
      enabled: True
      retain:
        days: 0
      events:
        retain:
          default: 5
          mode: motion
#        required_zones:
#          - front_driveway_1_close_in
#          - front_driveway_1_whole_area
        pre_capture: 5
        post_capture: 15

  front_driveway_1:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.15:554/user=admin&password=test&channel=6&stream=0.sdp?real_stream--rtp-caching=100
          roles:
            - detect
            - record
    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      track:
        - person
        - dog
        - bicycle
        - cat
        - car
      filters:
        car:
          min_area: 15000
        person:
          min_area: 500
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
#      required_zones:
#        - front_driveway_1_close_in
#        - front_driveway_1_whole_area
      crop: True
      height: 500
      retain:
        default: 3
 #   zones:
 #     front_driveway_1_close_in:
 #       coordinates: 0,480,429,480,443,335,460,173,263,133,88,120,36,212,0,279
 #       objects:
 #         - car
 #         - person
 #         - dog
 #         - cat
 #         - bicycle
 #     front_driveway_1_whole_area:
 #       coordinates: 383,52,497,105,569,316,575,480,261,480,0,480,0,225,148,57
 #       objects:
 #         - person
 #         - cat
 #         - dog
 #         - bicycle
    motion:
      mask:
        - 1211,102,1682,101,1679,62,1211,64
    record:
      enabled: True
      retain:
        days: 0
      events:
        retain:
          default: 5
          mode: motion
#        required_zones:
#          - front_driveway_1_close_in
#          - front_driveway_1_whole_area
        pre_capture: 5
        post_capture: 15

  front_driveway_2:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.15:554/user=admin&password=test&channel=8&stream=0.sdp?real_stream--rtp-caching=100
          roles:
            - detect
            - record
    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      track:
        - person
        - dog
        - bicycle
        - cat
        - car
      filters:
        car:
          min_area: 15000
        person:
          min_area: 500
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
#      required_zones:
#        - front_driveway_2_close_in
#        - front_driveway_2_whole_area
      crop: True
      height: 500
      retain:
        default: 3
#    zones:
#      front_driveway_2_close_in:
#        coordinates: 0,480,429,480,443,335,460,173,263,133,88,120,36,212,0,279
#        objects:
#          - car
#          - person
#          - dog
#          - cat
#          - bicycle
 #     front_driveway_2_whole_area:
 #       coordinates: 383,52,497,105,569,316,575,480,261,480,0,480,0,225,148,57
 #       objects:
 #         - person
 #         - cat
 #         - dog
 #         - bicycle
    motion:
      mask:
        - 1211,102,1682,101,1679,62,1211,64
        - 889,476,891,596,785,598,786,476
    record:
      enabled: True
      retain:
        days: 0
      events:
        retain:
          default: 5
          mode: motion
#        required_zones:
#          - front_driveway_2_close_in
#          - front_driveway_2_whole_area
        pre_capture: 5
        post_capture: 15

  front_porch:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.15:554/user=admin&password=test&channel=7&stream=0.sdp?real_stream--rtp-caching=100
          roles:
            - record
            - detect
    detect:
      width: 1920
      height: 1080
      fps: 5
      stationary:
        max_frames:
          objects:
            person: 100
    objects:
      track:
        - person
        - dog
        - bicycle
        - cat
  #    mask:
  #      - 0,480,198,480,200,449,0,451
  #      - 640,0,640,36,640,111,608,135,485,96,504,0
  #      - 439,200,441,324,512,320,532,261,554,188,540,129,451,134
  #      - 370,0,356,100,291,79,292,0
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: false
      crop: True
      height: 500
#      required_zones:
#        - front_porch_close_in
      retain:
        default: 5
    motion:
      mask:
        - 1211,102,1682,101,1679,62,1211,64
    #zones:
    #  front_porch_close_in:
    #    coordinates: 45,480,362,480,640,480,640,213,554,145,525,328,466,343,408,272,348,70,253,64,181,118,67,178
    record:
      enabled: True
      retain:
        days: 0
        mode: active_objects
      events:
        retain:
          default: 4
          mode: active_objects
#        required_zones:
#         - front_porch_close_in
        pre_capture: 5
        post_capture: 15

  back_porch:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.15:554/user=admin&password=test&channel=1&stream=0.sdp?real_stream--rtp-caching=100
          roles:
            - detect
            - record
    detect:
      width: 1920
      height: 1080
      fps: 6
    objects:
      track:
        - person
        - dog
        - cat
        - bird
        - mouse
      filters:
        cat:
          min_score: 0.3
          threshold: 0.5
        dog:
          min_score: 0.3
          threshold: 0.5
 #      person:
 #         mask:
 #           - 79,385,107,480,287,480,259,313,107,268
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 4
    motion:
      mask:
        - 1211,102,1682,101,1679,62,1211,64
    record:
      enabled: True
      retain:
        days: 0
      events:
        retain:
          default: 4
          mode: motion
        pre_capture: 5
        post_capture: 15

  diningroom:
    ffmpeg:
      inputs:
        - path: rtsp://admin:test@192.168.50.227/live0
          roles:
            - detect
            - record
    detect:
      width: 1920
      height: 1080
      fps: 6
    objects:
      track:
        - person
        - dog
        - cat
        - bird
        - mouse
      filters:
        cat:
          min_score: 0.3
          threshold: 0.5
        dog:
          min_score: 0.3
          threshold: 0.5
#       person:
#          mask:
#            - 79,385,107,480,287,480,259,313,107,268
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 4
    motion:
      mask:
        - 1382,58,1872,55,1868,8,1382,7
    record:
      enabled: True
      retain:
        days: 0
      events:
        retain:
          default: 4
          mode: motion
        pre_capture: 5
        post_capture: 15

  kitchen:
    ffmpeg:
      inputs:
        - path: rtsp://admin:test@192.168.50.70/live0
          roles:
            - detect
            - record
    detect:
      width: 1920
      height: 1080
      fps: 6
    objects:
      track:
        - person
        - dog
        - cat
        - bird
        - mouse
      filters:
        cat:
          min_score: 0.3
          threshold: 0.5
        dog:
          min_score: 0.3
          threshold: 0.5
#       person:
#          mask:
#            - 79,385,107,480,287,480,259,313,107,268
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 4
    motion:
      mask:
        - 1382,58,1872,55,1868,8,1382,7
    record:
      enabled: True
      retain:
        days: 0
      events:
        retain:
          default: 4
          mode: motion
        pre_capture: 5
        post_capture: 15

  garage:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.15:554/user=admin&password=test&channel=2&stream=0.sdp?real_stream--rtp-caching=100
          roles:
            - detect
            - record
    detect:
      width: 1920
      height: 1080
      fps: 6
    objects:
      track:
        - person
        - dog
        - cat
        - bird
        - mouse
      filters:
        cat:
          min_score: 0.3
          threshold: 0.5
        dog:
          min_score: 0.3
          threshold: 0.5
#       person:
#          mask:
#            - 79,385,107,480,287,480,259,313,107,268
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 4
    motion:
      mask:
        - 1211,102,1682,101,1679,62,1211,64
    record:
      enabled: True
      retain:
        days: 0
      events:
        retain:
          default: 4
          mode: motion
        pre_capture: 5
        post_capture: 15

  greenhouse:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.50.15:554/user=admin&password=test&channel=3&stream=0.sdp?real_stream--rtp-caching=100
          roles:
            - detect
            - record
    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      track:
        - person
        - dog
        - bicycle
        - cat
    snapshots:
      enabled: true
      timestamp: false
      bounding_box: true
      retain:
        default: 3
    record:
      enabled: True
      retain:
        days: 0
      events:
        retain:
          default: 4
          mode: active_objects
        pre_capture: 5
        post_capture: 15
    motion:
      mask:
        - 1211,102,1682,101,1679,62,1211,64

detectors:
  coral:
    type: edgetpu
    device: usb

rtmp:
  enabled: false

birdseye:
  enabled: True
  width: 1280
  height: 720
  quality: 1
  mode: objects

live:
  height: 640
  quality: 1

Relevant log output

My docker-compose file:

version: '3.9'
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: blakeblackshear/frigate:0.12.0-beta2
    shm_size: '256mb'
    devices:
      - /dev/bus/usb:/dev/bus/usb
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config/frigate.yml:/config/config.yml:rw
      - ./media/:/media/frigate
#      - /mnt/footage:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554"
      - "1984:1984"
#    environment:
#      FRIGATE_RTSP_PASSWORD: password

FFprobe output from your camera

na

Frigate stats

{"back_porch":{"camera_fps":6.0,"capture_pid":314,"detection_fps":0.0,"ffmpeg_pid":331,"pid":288,"process_fps":6.0,"skipped_fps":0.0},"cpu_usages":{"%Cpu(s):":{"cpu":"id,","mem":"0.0"},"1":{"cpu":"0.0","mem":"0.0"},"112":{"cpu":"0.3","mem":"0.1"},"113":{"cpu":"0.0","mem":"0.1"},"114":{"cpu":"0.0","mem":"0.1"},"115":{"cpu":"0.0","mem":"0.1"},"15":{"cpu":"0.0","mem":"0.0"},"1683":{"cpu":"0.0","mem":"0.0"},"1687":{"cpu":"0.3","mem":"0.0"},"1695":{"cpu":"1.7","mem":"0.4"},"17":{"cpu":"0.0","mem":"0.0"},"24":{"cpu":"0.0","mem":"0.0"},"25":{"cpu":"0.0","mem":"0.0"},"263":{"cpu":"0.0","mem":"1.0"},"269":{"cpu":"0.7","mem":"0.1"},"270":{"cpu":"2.3","mem":"1.6"},"272":{"cpu":"4.0","mem":"1.4"},"279":{"cpu":"1.0","mem":"1.5"},"280":{"cpu":"0.0","mem":"0.2"},"281":{"cpu":"0.7","mem":"1.4"},"282":{"cpu":"0.0","mem":"0.2"},"283":{"cpu":"0.7","mem":"1.6"},"284":{"cpu":"0.0","mem":"0.2"},"285":{"cpu":"0.7","mem":"1.5"},"286":{"cpu":"0.0","mem":"0.2"},"287":{"cpu":"4.3","mem":"1.4"},"288":{"cpu":"1.0","mem":"1.4"},"289":{"cpu":"0.0","mem":"0.2"},"290":{"cpu":"0.3","mem":"1.4"},"291":{"cpu":"0.7","mem":"1.4"},"292":{"cpu":"0.0","mem":"0.2"},"293":{"cpu":"1.0","mem":"1.4"},"294":{"cpu":"1.0","mem":"1.4"},"295":{"cpu":"0.0","mem":"0.2"},"296":{"cpu":"3.7","mem":"1.4"},"297":{"cpu":"0.0","mem":"0.2"},"298":{"cpu":"3.3","mem":"1.4"},"302":{"cpu":"3.3","mem":"1.4"},"304":{"cpu":"18.6","mem":"0.9"},"305":{"cpu":"0.0","mem":"0.2"},"306":{"cpu":"3.7","mem":"1.4"},"307":{"cpu":"15.3","mem":"0.8"},"308":{"cpu":"3.3","mem":"1.4"},"313":{"cpu":"14.6","mem":"1.1"},"314":{"cpu":"5.6","mem":"1.4"},"315":{"cpu":"0.0","mem":"0.2"},"317":{"cpu":"4.7","mem":"1.4"},"324":{"cpu":"4.7","mem":"1.4"},"328":{"cpu":"11.0","mem":"1.0"},"329":{"cpu":"0.0","mem":"0.2"},"33":{"cpu":"0.0","mem":"0.0"},"331":{"cpu":"19.3","mem":"1.3"},"332":{"cpu":"11.3","mem":"1.0"},"333":{"cpu":"4.0","mem":"1.4"},"338":{"cpu":"4.3","mem":"1.4"},"345":{"cpu":"16.6","mem":"1.0"},"348":{"cpu":"15.6","mem":"1.2"},"352":{"cpu":"16.3","mem":"1.3"},"360":{"cpu":"15.0","mem":"1.2"},"72":{"cpu":"0.0","mem":"0.0"},"73":{"cpu":"0.0","mem":"0.0"},"74":{"cpu":"0.0","mem":"0.0"},"75":{"cpu":"0.0","mem":"0.0"},"76":{"cpu":"0.0","mem":"0.0"},"77":{"cpu":"0.0","mem":"0.0"},"79":{"cpu":"0.0","mem":"0.2"},"80":{"cpu":"0.0","mem":"0.0"},"81":{"cpu":"0.0","mem":"0.0"},"82":{"cpu":"0.0","mem":"0.2"},"87":{"cpu":"0.0","mem":"0.0"},"89":{"cpu":"17.3","mem":"4.0"},"MiB":{"cpu":"5743.3","mem":"avail"},"PID":{"cpu":"%CPU","mem":"%MEM"},"Tasks:":{"cpu":"stopped,","mem":"0"},"top":{"cpu":"average:","mem":"4.67,"}},"detection_fps":5.1,"detectors":{"coral":{"detection_start":0.0,"inference_speed":10.03,"pid":270}},"diningroom":{"camera_fps":6.0,"capture_pid":317,"detection_fps":0.0,"ffmpeg_pid":328,"pid":290,"process_fps":6.0,"skipped_fps":0.0},"front_driveway_1":{"camera_fps":5.0,"capture_pid":302,"detection_fps":0.0,"ffmpeg_pid":313,"pid":283,"process_fps":5.0,"skipped_fps":0.0},"front_driveway_2":{"camera_fps":5.1,"capture_pid":306,"detection_fps":0.0,"ffmpeg_pid":345,"pid":285,"process_fps":5.1,"skipped_fps":0.0},"front_porch":{"camera_fps":5.1,"capture_pid":308,"detection_fps":5.1,"ffmpeg_pid":352,"pid":287,"process_fps":5.1,"skipped_fps":0.0},"garage":{"camera_fps":6.0,"capture_pid":333,"detection_fps":0.0,"ffmpeg_pid":360,"pid":293,"process_fps":6.0,"skipped_fps":0.0},"greenhouse":{"camera_fps":5.1,"capture_pid":338,"detection_fps":0.0,"ffmpeg_pid":348,"pid":294,"process_fps":5.1,"skipped_fps":0.0},"greenhouse_2":{"camera_fps":5.1,"capture_pid":298,"detection_fps":0.0,"ffmpeg_pid":304,"pid":281,"process_fps":5.1,"skipped_fps":0.0},"kitchen":{"camera_fps":6.0,"capture_pid":324,"detection_fps":0.0,"ffmpeg_pid":332,"pid":291,"process_fps":6.0,"skipped_fps":0.0},"license_plate":{"camera_fps":5.1,"capture_pid":296,"detection_fps":0.0,"ffmpeg_pid":307,"pid":279,"process_fps":5.1,"skipped_fps":0.0},"service":{"latest_version":"0.11.1","storage":{"/dev/shm":{"free":232.6,"mount_type":"tmpfs","total":268.4,"used":35.8},"/media/frigate/clips":{"free":172408.9,"mount_type":"ext4","total":248215.2,"used":64583.2},"/media/frigate/recordings":{"free":172408.9,"mount_type":"ext4","total":248215.2,"used":64583.2},"/tmp/cache":{"free":922.6,"mount_type":"tmpfs","total":1000.0,"used":77.4}},"temperatures":{},"uptime":686,"version":"0.12.0-0dbf909"}}

Operating system

HassOS

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

dahua

Any other information that may be helpful

No response

NickM-27 commented 1 year ago

I am trying to get the birdseye rtsp view on my tv, but whenever i try to pull it (i use webrtc dash cast to put the rtsp stream on it) i see this in the frigate logs:

You haven't enabled birdseye restream so naturally it won't work

In addition, my camera entities from frigate that i stream or record are still are as slow as 30seconds behind, which i don't understand. When i used generic camera entities i'm pretty sure it was close to realtime. Maybe it also has something to do with this error.

what are you using to stream? I have used the dash cast and it is almost literally real time. If you are using HA stream function directly then it is of course still using HLS which is far behind real time

atv2016 commented 1 year ago

Argh. Do you have a link or example config for how to enable it? Is it put under restream? Sorry if i'm being ignorant. And i'm guessing the only way of seeing this is by using RTSP right, no way of 'casting a birdseye camera entity (i believe one can do this in the frigate card by just filling in camera.birdseye?)

So i used the camera.stream or camera.record with the camera entity. I used to use webrtc with RTSP (which was near realtime) but i haven't tried it with the camera entities, i will try that.

NickM-27 commented 1 year ago

Argh. Do you have a link or example config for how to enable it? Is it put under restream? Sorry if i'm being ignorant.

It is covered in the restream docs https://deploy-preview-4055--frigate-docs.netlify.app/configuration/restream#birdseye-restream

And i'm guessing the only way of seeing this is by using RTSP right, no way of 'casting a birdseye camera entity (i believe one can do this in the frigate card by just filling in camera.birdseye?)

you can cast birdseye using webrtc via dash cast just like any other stream

atv2016 commented 1 year ago

Many thanks, i'll have a good read through of that as i do want to minimise the streams.

I tried to use webrtc dash cast to cast the rtsp stream to my chromecast, and it works but it seems to be frozen almost. It works fine in VLC. If i change the settings in the frigate webpage for birdseye to go from MSE to webrtc or JPEG, would that affect the rtsp stream as well? Maybe that will have a better effect on the chromecast. Or should i downscale the resolution somewhat.

I currently have this: birdseye: enabled: True width: 1920 height: 1080 quality: 1 mode: continuous

restream: birdseye: True jsmpeg: height: 640 quality: 1

NickM-27 commented 1 year ago

If i change the settings in the frigate webpage for birdseye to go from MSE to webrtc or JPEG, would that affect the rtsp stream as well?

No the setting there is just for viewing, it has no affect on how it works for other consumers.

atv2016 commented 1 year ago

Hey Nick, thanks. Ok, i guess i'll have to try with the newest chromecast. This is v1 chromecast, so might not be able to display it, although it supports 1080P.

NickM-27 commented 1 year ago

I think this issue can be closed? Seems it is working fine in VLC so on frigate side things are working as expected

atv2016 commented 1 year ago

Yes, i suppose so. Thank you for your help!