search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.21k stars 1.76k forks source link

Clip playback stutters. Camera feed stutters. Only through Frigate. FFMPEG Issue? #1499

Closed craigmdennis closed 3 years ago

craigmdennis commented 3 years ago

Describe the bug

When clips are recorded, they seems to stutter. They'll play fine for a few seconds and then freeze for a second, then the playpack video speed increases to catch up with where the video would have been had it not stuttered. This happens on both camera feeds. Video at the bottom.

I have a Reolink 410W and a Foscam PTZ. I have been running frigate with a Coral PCIe in HassOS on an Intel NUC.

Detection works as expected.

I have since tried this in a more powerful/newer Intel NUC7i3BNK (the one recommended in the docs) and this still happens. I switched to a USB Coral and it still happens.

When viewing the camera directly through the feed created by the HassOS Frigate addon the stuttering is actually part of the feed.

When I view the feed using VLC there is no stutter. When I view the feed generated by Synology Surveillance station there is no stutter.

I have tried deactivating the cameras in Surveillance station in case the camera resources couldn't handle so many connections. No effect. They still stutter.

I suspect it might be FFMPEG configuration but I have zero about those params. I've copied them from the docs for my cameras and hardware acceleration.

Config file Include your full config file wrapped in triple back ticks.

mqtt:
  host: 192.168.1.***
  user: mqttuser
  password: PASSWORD
  topic_prefix: frigate

cameras:
  garage:
    ffmpeg:
      hwaccel_args:
        - -hwaccel
        - vaapi
        - -hwaccel_device
        - /dev/dri/renderD128
        - -hwaccel_output_format
        - yuv420p
      inputs:
        - path: rtsp://foscam_garage:PASSWORD@192.168.1.***:88/videoSub
          roles:
            - detect
        - path: rtsp://foscam_garage:PASSWORD@192.168.1.***:88/videoMain
          roles: 
            - clips
            - rtmp
    width: 1280
    height: 720
    fps: 5
    objects:
      track:
        - person
        - dog
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 7
        objects:
          person: 14
    clips:
      enabled: true
      retain:
        default: 2
        objects:
          person: 7
  driveway:
    ffmpeg: 
      hwaccel_args:
        - -hwaccel
        - vaapi
        - -hwaccel_device
        - /dev/dri/renderD128
        - -hwaccel_output_format
        - yuv420p
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
        - -rw_timeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      inputs:
        - path: rtmp://192.168.1.***/bcs/channel0_sub.bcs?channel=0&stream=0&user=frigate&password=PASSWORD
          roles:
            - detect
        - path: rtmp://192.168.1.***/bcs/channel0_main.bcs?channel=0&stream=0&user=frigate&password=PASSWORD
          roles:
            - rtmp
            - clips
    width: 640
    height: 352
    fps: 7
    objects:
      track:
        - person
        - dog
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 7
        objects:
          person: 14
    clips:
      enabled: true
      retain:
        default: 2
        objects:
          person: 7
    zones:
      cars:
        coordinates: 214,21,308,80,378,0,640,0,640,352,0,352,0,177,124,121
      neighbours:
        coordinates: 198,0,0,0,0,141

detectors:
  coral:
    type: edgetpu
    device: usb

Frigate stats

{
  "cameras": {
    "driveway": {
      "best_image_timeout": 60,
      "clips": {
        "enabled": true,
        "objects": null,
        "post_capture": 5,
        "pre_capture": 5,
        "required_zones": [],
        "retain": {
          "default": 2,
          "objects": {
            "person": 7
          }
        }
      },
      "detect": {
        "enabled": true,
        "max_disappeared": 35
      },
      "ffmpeg_cmds": [
        {
          "cmd": "ffmpeg -hide_banner -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rw_timeout 5000000 -use_wallclock_as_timestamps 1 -i rtmp://192.168.1.182/bcs/channel0_sub.bcs?channel=0&stream=0&user=frigate&password=ripple4escallop_neurosis*SENT -r 7 -f rawvideo -pix_fmt yuv420p pipe:",
          "roles": [
            "detect"
          ]
        },
        {
          "cmd": "ffmpeg -hide_banner -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rw_timeout 5000000 -use_wallclock_as_timestamps 1 -i rtmp://192.168.1.182/bcs/channel0_main.bcs?channel=0&stream=0&user=frigate&password=ripple4escallop_neurosis*SENT -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/driveway-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/driveway",
          "roles": [
            "rtmp",
            "clips"
          ]
        }
      ],
      "fps": 7,
      "frame_shape": [
        352,
        640
      ],
      "height": 352,
      "motion": {
        "contour_area": 100,
        "delta_alpha": 0.2,
        "frame_alpha": 0.2,
        "frame_height": 58,
        "mask": null,
        "threshold": 25
      },
      "mqtt": {
        "bounding_box": true,
        "crop": true,
        "enabled": true,
        "height": 270,
        "required_zones": [],
        "timestamp": true
      },
      "name": "driveway",
      "objects": {
        "filters": {
          "dog": {
            "mask": [],
            "max_area": 24000000,
            "min_area": 0,
            "min_score": 0.5,
            "threshold": 0.7
          },
          "person": {
            "mask": [],
            "max_area": 24000000,
            "min_area": 0,
            "min_score": 0.5,
            "threshold": 0.7
          }
        },
        "mask": null,
        "track": [
          "person",
          "dog"
        ]
      },
      "record": {
        "enabled": false,
        "retain_days": 30
      },
      "rtmp": {
        "enabled": true
      },
      "snapshots": {
        "bounding_box": true,
        "crop": false,
        "enabled": true,
        "height": null,
        "required_zones": [],
        "retain": {
          "default": 7,
          "objects": {
            "person": 14
          }
        },
        "timestamp": true
      },
      "width": 640,
      "zones": {
        "cars": {
          "coordinates": "214,21,308,80,378,0,640,0,640,352,0,352,0,177,124,121",
          "filters": {}
        },
        "neighbours": {
          "coordinates": "198,0,0,0,0,141",
          "filters": {}
        }
      }
    },
    "garage": {
      "best_image_timeout": 60,
      "clips": {
        "enabled": true,
        "objects": null,
        "post_capture": 5,
        "pre_capture": 5,
        "required_zones": [],
        "retain": {
          "default": 2,
          "objects": {
            "person": 7
          }
        }
      },
      "detect": {
        "enabled": true,
        "max_disappeared": 25
      },
      "ffmpeg_cmds": [
        {
          "cmd": "ffmpeg -hide_banner -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://foscam_garage:ukD8HaXmT8XZ@192.168.1.101:88/videoSub -r 5 -f rawvideo -pix_fmt yuv420p pipe:",
          "roles": [
            "detect"
          ]
        },
        {
          "cmd": "ffmpeg -hide_banner -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://foscam_garage:ukD8HaXmT8XZ@192.168.1.101:88/videoMain -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/garage-%Y%m%d%H%M%S.mp4 -c copy -f flv rtmp://127.0.0.1/live/garage",
          "roles": [
            "clips",
            "rtmp"
          ]
        }
      ],
      "fps": 5,
      "frame_shape": [
        720,
        1280
      ],
      "height": 720,
      "motion": {
        "contour_area": 100,
        "delta_alpha": 0.2,
        "frame_alpha": 0.2,
        "frame_height": 120,
        "mask": null,
        "threshold": 25
      },
      "mqtt": {
        "bounding_box": true,
        "crop": true,
        "enabled": true,
        "height": 270,
        "required_zones": [],
        "timestamp": true
      },
      "name": "garage",
      "objects": {
        "filters": {
          "dog": {
            "mask": [],
            "max_area": 24000000,
            "min_area": 0,
            "min_score": 0.5,
            "threshold": 0.7
          },
          "person": {
            "mask": [],
            "max_area": 24000000,
            "min_area": 0,
            "min_score": 0.5,
            "threshold": 0.7
          }
        },
        "mask": null,
        "track": [
          "person",
          "dog"
        ]
      },
      "record": {
        "enabled": false,
        "retain_days": 30
      },
      "rtmp": {
        "enabled": true
      },
      "snapshots": {
        "bounding_box": true,
        "crop": false,
        "enabled": true,
        "height": null,
        "required_zones": [],
        "retain": {
          "default": 7,
          "objects": {
            "person": 14
          }
        },
        "timestamp": true
      },
      "width": 1280,
      "zones": {}
    }
  },
  "clips": {
    "max_seconds": 300,
    "retain": {
      "default": 10,
      "objects": {}
    },
    "tmpfs_cache_size": ""
  },
  "database": {
    "path": "/media/frigate/clips/frigate.db"
  },
  "detectors": {
    "coral": {
      "device": "usb",
      "num_threads": 3,
      "type": "edgetpu"
    }
  },
  "environment_vars": {},
  "logger": {
    "default": "INFO",
    "logs": {}
  },
  "model": {
    "height": 320,
    "width": 320
  },
  "mqtt": {
    "client_id": "frigate",
    "host": "192.168.1.***",
    "port": 1883,
    "stats_interval": 60,
    "topic_prefix": "frigate",
    "user": "mqttuser"
  },
  "snapshots": {
    "retain": {
      "default": 10,
      "objects": {}
    }
  }
}

Computer Hardware

Camera Info:

Additional context

Clip (trimmed):

https://user-images.githubusercontent.com/725067/129099066-02f75bce-0495-4ad0-a3d7-688b1bf63ddd.mp4

craigmdennis commented 3 years ago

Possibly related, when viewing the main stream in the LoveLace UI it seems to buffer. Again, this doesn't happen when using the feed from Synology Surveillance station.

Ignore the wiggly mouse movement, it was to keep the playback controls visible.

2021-08-11_22 23 10_Firefox_j7G2IUUz

blakeblackshear commented 3 years ago

Try these args: https://github.com/blakeblackshear/frigate/issues/1467#issue-962453322

craigmdennis commented 3 years ago

Yes, those options resolved my issue. The options in my config were copied from the docs related to the camera specifically. So it might be worth inluding these options as well? If so, I'd be happy to submit a PR.