Error with camera setup in Frigate

man-jiteshm-sportsbet commented 1 week ago

Hey @felipecrs

i followed your guide to setup my DAHU VTO2202F-p-S2 in frigate but i get an error as below

2024-11-04 15:14:26.087502749  [2024-11-04 15:14:26] frigate.video                  ERROR   : doorbell: ffmpeg process is not running. exiting capture thread...
2024-11-04 15:14:26.346231567  
2024-11-04 15:14:40.984917118  [2024-11-04 15:14:40] watchdog.doorbell              ERROR   : Ffmpeg process crashed unexpectedly for doorbell.
2024-11-04 15:14:40.985045382  [2024-11-04 15:14:40] watchdog.doorbell              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-11-04 15:14:40.985213117  [2024-11-04 15:14:40] ffmpeg.doorbell.detect         ERROR   : rtsp://127.0.0.1:8554/doorbell: Invalid data found when processing input
2024-11-04 15:14:46.109904488  [2024-11-04 15:14:46] frigate.video                  ERROR   : doorbell: Unable to read frames from ffmpeg process.

my config is this

go2rtc:
  streams:
    doorbell:
      # H264, PCMA, 2-way audio
      - rtsp://admin:********@192.168.30.108/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif#backchannel=1
      # AAC
      - ffmpeg:doorbell#audio=aac

cameras:
  doorbell:
    ffmpeg:
      inputs:
      - path: rtsp://127.0.0.1:8554/doorbell
        input_args: preset-rtsp-restream-low-latency
        roles:
        - detect

i have tried it with rtmp disabled/enabled both. also tried using - rtsp://127.0.0.1:8554/doorbell?audio=aac

felipecrs commented 1 week ago

Your go2rtc configuration is very different from the one I recommend.

https://github.com/felipecrs/dahua-vto-on-home-assistant/blob/master/frigate%2Ffrigate.yaml

felipecrs commented 1 week ago

What version of Frigate are you using?

man-jiteshm-sportsbet commented 1 week ago

frigate: System 0.13.2-6476f8a

man-jiteshm-sportsbet commented 1 week ago

Your go2rtc configuration is very different from the one I recommend.

https://github.com/felipecrs/dahua-vto-on-home-assistant/blob/master/frigate%2Ffrigate.yaml

i've only setup 1 stream for detect as i don't want to keep recording the video. i had tried your config exactly as is, and i was getting errors so i had to modify it a bit. i thought it might be due to version differences. Which part of it are you mainly concerned about?

could answering the call on DMSS app cause issues?

man-jiteshm-sportsbet commented 1 week ago

looking through your file closer, is this deliberate or accidental but L7 and L12 look to be referencing the wrong stream name.

https://github.com/felipecrs/dahua-vto-on-home-assistant/blob/master/frigate%2Ffrigate.yaml#L7,L12

felipecrs commented 1 week ago

No, they are referencing the right stream name. I do that to avoid having to transcode.

But anyway, you are mainly missing the fix_vto_codecs.sh.

man-jiteshm-sportsbet commented 1 week ago

But anyway, you are mainly missing the fix_vto_codecs.sh.

yes i purposely left that out because i am running frigate on a separate docker container. it is not running as an add-on

i might have resolved the errors by only using 1 stream for now and everything seems to be working but i randomly get an error in the frigate logs as below. have you seen this?

i checked the Dahua web interface and ONVIF is turned on

2024-11-05 00:18:59.821783186  [2024-11-05 00:18:59] frigate.ptz.onvif              ERROR   : Onvif is not setup for doorbell
2024-11-05 00:23:48.299395669  [2024-11-05 00:23:48] frigate.ptz.onvif              ERROR   : Onvif is not setup for doorbell
2024-11-05 00:34:26.406347173  [2024-11-05 00:34:26] frigate.ptz.onvif              ERROR   : Onvif is not setup for doorbell
2024-11-05 10:48:52.132484821  [2024-11-05 10:48:52] frigate.ptz.onvif              ERROR   : Onvif is not setup for doorbell

felipecrs commented 1 week ago

This onvif error is completely normal to have, and it's "caused" by the Frigate card.

https://github.com/blakeblackshear/frigate/issues/7094

felipecrs commented 1 week ago

i might have resolved the errors by only using 1 stream for now and everything seems to be working

I believe this is a known bug of go2rtc but can't remember which one exactly now. Anyway, my recommended config has no ffmpeg: and is not affected by this issue.

yes i purposely left that out because i am running frigate on a separate docker container. it is not running as an add-on

This conclusion is incorrect, it is meant to be used regardless of Frigate being ran as a container or as an add-on. When running Frigate via docker, you can copy the script to the same directory where your frigate.yaml is, and then go2rtc will be able to find it as /config/fix_vto_codecs.sh.

I would suggest you give it a try, it's the best optimized configuration.

felipecrs commented 1 week ago

Also, I understand that you don't want to record video, but having an AAC track is still good for compatibility purposes. For example, Home Assistant HLS player can only play AAC audio. The same for when you are casting the camera to a Chromecast/Nest Hub. Also, I believe the WebRTC card may use AAC to connect in MSE in case WebRTC stream isn't possible.

felipecrs commented 1 week ago

Finally, remember that VTO 2-way audio doesn't work unless the codecs are properly set, and this is another thing fix_vto_codecs.sh does as well.

man-jiteshm-sportsbet commented 1 week ago

I believe this is a known bug of go2rtc but can't remember which one exactly now. Anyway, my recommended config has no ffmpeg: and is not affected by this issue.

so i tried with your config, and i get an error when using - rtsp://127.0.0.1:8554/doorbell_hd?audio=aac

2024-11-05 18:09:51.234115490  [ERROR:0@17.337] global cap.cpp:164 open VIDEOIO(CV_IMAGES): raised OpenCV exception:
2024-11-05 18:09:51.234122025  
2024-11-05 18:09:51.234129651  OpenCV(4.7.0) /io/opencv/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): rtsp://127.0.0.1:8554/doorbell in function 'icvExtractPattern'
2024-11-05 18:09:58.311107077  [2024-11-05 18:09:58] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2024-11-05 18:10:01.478481220  [2024-11-05 18:10:01] frigate.util.services          ERROR   : Unable to poll intel GPU stats: Failed to initialize PMU! (Invalid argument)
2024-11-05 18:10:01.478488353  
2024-11-05 18:10:01.709992724  [2024-11-05 18:10:01] frigate.video                  ERROR   : doorbell: Unable to read frames from ffmpeg process.
2024-11-05 18:10:01.710167210  [2024-11-05 18:10:01] frigate.video                  ERROR   : doorbell: ffmpeg process is not running. exiting capture thread...

When running Frigate via docker, you can copy the script to the same directory where your frigate.yaml is, and then go2rtc will be able to find it as /config/fix_vto_codecs.sh.

i tried this too and it was throwing errors in the frigate startup, so i had to remove it and ran the script locally for now to apply the right settings on the vto until it's rebooted.

felipecrs commented 1 week ago

i tried this too and it was throwing errors in the frigate startup

Which error? Maybe you just need to chmod +x path/to/fix_vto_codecs.sh.

and i get an error when using

Please share your config again, redacting your password.

man-jiteshm-sportsbet commented 1 week ago

this is my frigate config

ui:
  use_experimental: false

# Include all cameras by default in Birdseye view
birdseye:
  enabled: True
  mode: continuous

mqtt:
  host: xxx
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: xxx
  password: xxx

rtmp:
  enabled: false

detectors:
  coral:
    type: edgetpu
    device: usb

ffmpeg:
  hwaccel_args: preset-vaapi
  output_args:
    record: preset-record-generic-audio-copy
  #input_args: preset-rtsp-restream

go2rtc:
  streams:
    front_main:
      - rtsp://xxx:xxx@192.168.xx.xx/cam/realmonitor?channel=1&subtype=0
      - ffmpeg:front_main#audio=opus       # <- copy of the stream which transcodes audio to opus
    front_sub:
      - rtsp://xxx:xxx@192.168.xx.xx/cam/realmonitor?channel=1&subtype=2
      - ffmpeg:front_sub#audio=opus       # <- copy of the stream which transcodes audio to opus
    doorbell:
      # H264, PCMA, 2-way audio
      - rtsp://xxx:xxx@192.168.xx.xx/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif#backchannel=1
      # AAC
      - ffmpeg:doorbell#audio=aac
    #doorbell_hd:
    #  # H264, AAC
    #  - rtsp://xxx:xxx@192.168.xx.xx/cam/realmonitor?channel=1&subtype=1#backchannel=0
    #  # PCMA
    #  - ffmpeg:doorbell_hd#audio=pcma
  webrtc:
    candidates:
    - 192.168.xx.xx:8555
    - stun:8555

snapshots:
  required_zones:
  - pathway
  - driveway
  enabled: true
  bounding_box: true
  clean_copy: true
  retain:
    default: 30

record:
  sync_recordings: true
  enabled: true
  events:
    pre_capture: 5
    post_capture: 5
    retain:
      default: 30
      mode: active_objects
  retain:
    days: 14
    mode: all

cameras:
  doorbell:
    ffmpeg:
      inputs:
      - path: rtsp://127.0.0.1:8554/doorbell
        input_args: preset-rtsp-restream-low-latency
        roles:
        - detect
        - rtmp
      #- path: rtsp://127.0.0.1:8554/doorbell_hd
      #  input_args: preset-rtsp-restream-low-latency
      #  roles:
      #    - record
  front:
    ffmpeg:
      inputs:
      - path: rtsp://127.0.0.1:8554/front_main     # <--- the name here must match the name of the camera in restream
        input_args: preset-rtsp-restream
        roles:
        - record
      - path: rtsp://127.0.0.1:8554/front_sub     # <--- the name here must match the name of the camera_sub in restream
        input_args: preset-rtsp-restream
        roles:
        - detect
    live:
      stream_name: front_main
    objects:
      track:
      - person
      - car
      - bicycle
      - motorcycle
      - cat
      - dog
    record:
      events:
        required_zones:
        - driveway
        - pathway
    motion:
      mask:
        - 268,1018,678,301,1168,290,893,1080,442,1058
        - 531,331,78,938,0,890,0,335
        - 1920,0,1920,340,0,348,0,0
    zones:
      driveway:
        coordinates: 913,1080,1920,1080,1920,763,1573,382,1153,380
        objects:
        - person
        - car
        - bicycle
        - motorcycle
        - cat
        - dog
      pathway:
        coordinates: 635,360,242,1008,93,925,523,363
        objects:
        - person
        - bicycle
        - cat
        - dog

man-jiteshm-sportsbet commented 1 week ago

Which error? Maybe you just need to chmod +x path/to/fix_vto_codecs.sh.

i have already put chmod +x on the file. i am able to execute it from within the container

docker terminal:

root@06444ff2c51c:/config# ls -ltr fix_vto_codecs.sh 
-rwxr-xr-x 1 1027 users 964 Nov  3 20:13 fix_vto_codecs.sh
root@06444ff2c51c:/config# pwd
/config

frigate error when i put echo:/config/fix_vto_codecs.sh:

[INFO] Preparing Frigate...
[INFO] Starting Frigate...
[2024-11-07 20:00:29] frigate.app                    INFO    : Starting Frigate (0.13.2-6476f8a)
[rtsp @ 0x2e16f80] method DESCRIBE failed: 404 Not Found
[ERROR:0@18.221] global cap.cpp:164 open VIDEOIO(CV_IMAGES): raised OpenCV exception:

OpenCV(4.7.0) /io/opencv/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): rtsp://127.0.0.1:8554/doorbell in function 'icvExtractPattern'

[2024-11-07 20:00:48] peewee_migrate.logs            INFO    : Starting migrations
[2024-11-07 20:00:48] peewee_migrate.logs            INFO    : There is nothing to migrate
[2024-11-07 20:00:49] frigate.app                    INFO    : Recording process started: 333
[2024-11-07 20:00:49] frigate.app                    INFO    : go2rtc process pid: 90
[2024-11-07 20:00:49] detector.coral                 INFO    : Starting detection process: 343
[2024-11-07 20:00:49] frigate.app                    INFO    : Output process started: 345
[2024-11-07 20:00:49] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
[2024-11-07 20:00:51] frigate.app                    INFO    : Camera processor started for doorbell: 376
[2024-11-07 20:00:51] frigate.app                    INFO    : Camera processor started for front: 377
[2024-11-07 20:00:51] frigate.app                    INFO    : Capture process started for doorbell: 378
[2024-11-07 20:00:51] frigate.app                    INFO    : Capture process started for front: 380
[2024-11-07 20:00:52] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
[2024-11-07 20:00:56] frigate.video                  ERROR   : doorbell: Unable to read frames from ffmpeg process.
[2024-11-07 20:00:56] frigate.video                  ERROR   : doorbell: ffmpeg process is not running. exiting capture thread...
[2024-11-07 20:01:11] watchdog.doorbell              ERROR   : Ffmpeg process crashed unexpectedly for doorbell.
[2024-11-07 20:01:11] watchdog.doorbell              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
[2024-11-07 20:01:11] ffmpeg.doorbell.detect         ERROR   : rtsp://127.0.0.1:8554/doorbell: Invalid data found when processing input

man-jiteshm-sportsbet commented 1 week ago

@felipecrs my other issue is in setting up the frigate hass card. when answering the call, the overrides dont seem to work. the microphone is not enabled, thus not shown. i force enabled the microphone visibility in the menu setting, but when i click on it, it's not connected.

man-jiteshm-sportsbet commented 1 week ago

https://github.com/user-attachments/assets/0bdc7dcb-a796-4d34-88e1-d24e0fbfccdd

This is my code which i copied from your reference. i have the same camera name and input_boolean names. i am running frigate-hass-card 6.0.0 beta8

type: custom:frigate-card
cameras:
  - camera_entity: camera.doorbell
    live_provider: go2rtc
    go2rtc:
      modes:
        - webrtc
    image:
      refresh_seconds: 1
    dimensions:
      aspect_ratio: "16:9"
live:
  controls:
    builtin: false
  zoomable: true
  show_image_during_load: true
  draggable: false
  lazy_unload:
    - unselected
    - hidden
  microphone:
    always_connected: false
    disconnect_seconds: 120
    mute_after_microphone_mute_seconds: 120
  auto_unmute:
    - microphone
menu:
  buttons:
    snapshots:
      enabled: false
    fullscreen:
      enabled: false
    cameras:
      enabled: false
    live:
      enabled: false
    clips:
      enabled: false
    microphone:
      type: toggle
      priority: 49
      enabled: true
    mute:
      enabled: true
  style: outside
  position: top
  button_size: 85
  alignment: right
status_bar:
  style: none
profiles:
  - low-performance
overrides:
  - conditions:
      - condition: microphone
        connected: true
    merge:
      menu:
        buttons:
          microphone:
            enabled: true
elements:
  - type: custom:frigate-card-conditional
    conditions:
      - condition: state
        state: "on"
        entity: input_boolean.doorbell_calling
    elements:
      - type: custom:frigate-card-menu-icon
        icon: mdi:phone
        alignment: opposing
        priority: 39
        style:
          color: green
        tap_action:
          - action: perform-action
            data:
              entity_id: input_boolean.doorbell_calling
            perform_action: input_boolean.turn_off
          - action: custom:frigate-card-action
            frigate_card_action: microphone_unmute
  - type: custom:frigate-card-conditional
    conditions:
      - condition: state
        state: "off"
        entity: input_boolean.doorbell_calling
      - condition: microphone
        connected: false
    elements:
      - type: custom:frigate-card-menu-icon
        icon: mdi:phone-outgoing
        alignment: opposing
        priority: 39
        style:
          color: green
        tap_action:
          - action: custom:frigate-card-action
            frigate_card_action: microphone_unmute
  - type: custom:frigate-card-conditional
    conditions:
      - condition: state
        state: "on"
        entity: input_boolean.doorbell_calling
    elements:
      - type: custom:frigate-card-menu-icon
        icon: mdi:phone-ring
        alignment: opposing
        priority: 40
        style:
          color: red
        tap_action:
          - action: perform-action
            data:
              entity_id: input_boolean.doorbell_calling
            perform_action: input_boolean.turn_off
  - type: custom:frigate-card-conditional
    conditions:
      - condition: state
        state: "off"
        entity: input_boolean.doorbell_calling
      - condition: microphone
        connected: true
    elements:
      - type: custom:frigate-card-menu-icon
        icon: mdi:phone-hangup
        alignment: opposing
        priority: 40
        style:
          color: red
        tap_action:
          - action: custom:frigate-card-action
            frigate_card_action: microphone_disconnect
          - action: custom:frigate-card-action
            frigate_card_action: mute

felipecrs commented 1 week ago

frigate error when i put echo:/config/fix_vto_codecs.sh:

That's weird. I don't see any indication that the script is causing that issue. Maybe go2rtc logs would have a better hint.

As per the Frigate Card issue:

Are you using the latest 6.0 beta?
Does 2-way audio work from Frigate itself? It must work there first. If it does not work there, it will not work in HA either.

man-jiteshm-sportsbet commented 1 week ago

How do i get more logs to find out? It only happens when i add the script in the path.

For frigate, how do i test 2-way audio from within frigate?

Yes i am using 6.0.0-beta8

felipecrs commented 1 week ago

How do i get more logs to find out? It only happens when i add the script in the path.

Under Frigate -> System logs -> go2rtc

For frigate, how do i test 2-way audio from within frigate?

(Frigate 0.14)

man-jiteshm-sportsbet commented 1 week ago

i have the same doorbell as you (vto2202f-p-s2), so im assuming it should work. i will test it out

felipecrs commented 1 week ago

It definitely should. Make sure to update your firmware btw, latest one is V4.600.0000000.2.R.240802:

https://www.dahuasecurity.com/pt/products/All-Products/Video-Intercoms/IP-Series/Villa-Door-Station/Pro-Series/VTO2202F-P-S2

Also, make sure your Home Assistant is running with https. Two way audio only works on browsers if https.

man-jiteshm-sportsbet commented 1 week ago

is your frigate running as HA addon, i don't see that frigate in my sidebar.

man-jiteshm-sportsbet commented 1 week ago

Also, make sure your Home Assistant is running with https. Two way audio only works on browsers if https.

i am running it behind tailscale on http. i only access it from within the companion app. will that be an issue?

man-jiteshm-sportsbet commented 1 week ago

i enabled https via nabu casa, and the microphone started working. thank you @felipecrs

man-jiteshm-sportsbet commented 6 days ago

is it possible to have the card always loaded even when changing tabs? i have tried adding preload: true and also lazy_unload: [] but everytime i switch views and come back to the frigate hass card, the video has to load and it takes. a couple of seconds.

felipecrs commented 6 days ago

I don't know either, never tried.

felipecrs commented 6 days ago

I believe we took care of everything in this issue, please let me know otherwise.

man-jiteshm-sportsbet commented 6 days ago

yes we can resolve it for now, i need to figure out how to get that script added to the frigate.yaml

man-jiteshm-sportsbet commented 6 days ago

putting a note here for anyone else interested, since i am on chrome, i have added enabled chrome://flags/#unsafely-treat-insecure-origin-as-secure and added my local IP my http://192.168.xx.xx:8123/ to the list.

my microphone now works on my local network via http, while i use nabu casa over the internet and https. this protects me from loosing intercom access if the internet goes down

felipecrs / dahua-vto-on-home-assistant

Error with camera setup in Frigate #6