search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.21k stars 1.76k forks source link

[Support]: FFMPEG pinning CPU at 100% even though hwaccell is enabled #2781

Closed Sloth-on-meth closed 2 years ago

Sloth-on-meth commented 2 years ago

Describe the problem you are having

image

Version

0.9.4-26AE608

Frigate config file

mqtt:
  host: 192.168.178.79 
  user: mosquitto 
  password: kek123
  stats_interval: 10

cameras:
#
#  front:
#    mqtt:
#      bounding_box: True
#    zones:
#      autosam:
#        coordinates: 984,119,821,22,590,0,389,99,307,243,167,579,260,1445,1143,1451
#      voordeur:
#        coordinates: 2560,1920,2560,1836,2560,834,1185,1018,1218,1899
#      straat:
#        coordinates: 0,618,380,133,594,39,856,55,1322,33,1336,0,0,0
#    motion:
#      mask:
#        - 2336,1035,2560,1920,2560,1613,2560,0,1860,0,2073,369
#        - 0,1920,0,1287,328,1327,420,1920  
#    ffmpeg:
#      hwaccel_args: 
#        - -c:v 
#        - h264_cuvid
#      input_args:
#        - -avoid_negative_ts
#        - make_zero
#        - -fflags
#        - nobuffer+genpts+discardcorrupt
#        - -flags
#        - low_delay
#        - -strict
#        - experimental
#        - -analyzeduration
#        - 1000M
#        - -probesize
#        - 1000M
#        - -rw_timeout
#        - "5000000"
#      inputs:
#        - path: http://192.168.178.113/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=PASSWORD
#          roles:
#            - record
#            - rtmp
#        - path: http://192.168.178.113/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=PASSWORD
#          roles:
#            - detect
#    detect:
#      width: 640
#      height: 480
#      fps: 7
#    snapshots:
#      enabled: True
#      timestamp: True
#      bounding_box: True
#      crop: True
#
  back:
    mqtt:
      bounding_box: True
    zones:
      zone_0:
        coordinates: 2560,45,2537,1375,2171,1350,1802,1324,1786,1440,0,1440,0,728,1358,34
      egelhuis:
        coordinates: 1585,597,1645,384,1780,381,1786,629
    motion:
      mask:
        - 1805,1440,2560,1440,2560,1298,1801,1305
        - 227,0,0,287,0,133,0,0,48,0,411,0
        - 0,0,787,0,1847,0,1367,34,1276,60,496,141,169,301,0,382
        - 2249,555,2241,57,2108,51,2127,545
        - 1523,276,1539,268,1582,216,1458,178,1398,0,1293,33,1355,219
        - 0,1440,704,1440,937,1203,704,1124,518,1070
        - 0,732,185,602,365,1079,0,1390
    ffmpeg:
      hwaccel_args: 
        - -c:v 
        - h264_cuvid
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer+genpts+discardcorrupt
        - -flags
        - low_delay
        - -strict
        - experimental
        - -analyzeduration
        - 1000M
        - -probesize
        - 1000M
        - -rw_timeout
        - "5000000"
      inputs:
        - path: http://192.168.178.94/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=PASSWORD
          roles:
            - record
            - rtmp
        - path: http://192.168.178.94/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=PASSWORD
          roles:
            - detect
    detect:
      width: 640
      height: 480
      fps: 7
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: True

  front:
    detect:
      max_disappeared: 100
      # Optional: Frequency for running detection on stationary objects (default: 10x the frame rate)
     # stationary_interval: 50
      width: 2560
      height: 1920
      fps: 5

    zones:
      autosam:
        coordinates: 984,119,821,22,590,0,389,99,307,243,167,579,260,1445,1143,1451
      voordeur:
        coordinates: 2560,1920,2560,1836,2560,834,1185,1018,1218,1899
      straat:
        coordinates: 0,618,380,133,594,39,856,55,1322,33,1336,0,0,0
    motion:
      mask:
        - 2336,1035,2560,1920,2560,1613,2560,0,1860,0,2073,369
        - 0,1920,0,1287,328,1327,420,1920

    mqtt:
      bounding_box: True

    ffmpeg:
      input_args:
        - -c:v
        - h264_cuvid
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
        - -rw_timeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1  -c:a aac 

      inputs:
        - path: http://192.168.178.94/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=PASSWORD
          roles:
            - record
            - detect
            - rtmp
      hwaccel_args: 
        - -c:v 
        - h264_cuvid
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: True

  back:
    detect:
      max_disappeared: 100
      # Optional: Frequency for running detection on stationary objects (default: 10x the frame rate)
      #stationary_interval: 50
      width: 2560
      height: 1440
      fps: 5
    zones:
      zone_0:
        coordinates: 2560,45,2537,1375,2171,1350,1802,1324,1786,1440,0,1440,0,728,1358,34
      egelhuis:
        coordinates: 1585,597,1645,384,1780,381,1786,629
    motion:
      mask:
        - 1805,1440,2560,1440,2560,1298,1801,1305
        - 227,0,0,287,0,133,0,0,48,0,411,0
        - 0,0,787,0,1847,0,1367,34,1276,60,496,141,169,301,0,382
        - 2249,555,2241,57,2108,51,2127,545
        - 1523,276,1539,268,1582,216,1458,178,1398,0,1293,33,1355,219
        - 0,1440,704,1440,937,1203,704,1124,518,1070
        - 0,732,185,602,365,1079,0,1390
    mqtt:
      bounding_box: True
    ffmpeg:
      input_args:
        - -c:v
        - h264_cuvid
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
        - -rw_timeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
        - -c:a
        - aac
      output_args:
        record: -f segment -segment_time 10 -reset_timestamps 1 -segment_format mp4  -strftime 1 -c copy 
      hwaccel_args: 
        - -c:v 
        - h264_cuvid

      inputs:
        - path:  http://192.168.178.113/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=PASSWORD
          roles:
            - record
            - detect
            - rtmp

    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: True
    timestamp_style:
      # Optional: Position of the timestamp (default: shown below)
      #           "tl" (top left), "tr" (top right), "bl" (bottom left), "br" (bottom right)
      position: "tl"
      # Optional: Format specifier conform to the Python package "datetime" (default: shown below)
      #           Additional Examples:
      #             german: "%d.%m.%Y %H:%M:%S"
      format: "%d/%m/%Y %H:%M:%S"
      # Optional: Color of font
      color:
        # All Required when color is specified (default: shown below)
        red: 255
        green: 255
        blue: 255
      # Optional: Line thickness of font (default: shown below)
      thickness: 2

live:
  # Optional: Set the height of the live stream. (default: 720)
  # This must be less than or equal to the height of the detect stream. Lower resolutions
  # reduce bandwidth required for viewing the live stream. Width is computed to match known aspect ratio.

  # Optional: Set the encode quality of the live stream (default: shown below)
  # 1 is the highest quality, and 31 is the lowest. Lower quality feeds utilize less CPU resources.
  quality: 20
  height: 1440
#  backdoor:
#    zones:
#      voordeur:
#        coordinates: 1280,398,826,720,0,720,0,470,124,483,499,454,515,252,806,0,1280,0
#    motion:
#      mask:
#        - 311,720,259,210,342,130,760,127,921,79,936,0,0,0,0,720
#        - 311,720,257,202,341,0,0,0,0,720
#    fps: 5
#    ffmpeg:
#      input_args:
#        - -c:v
#        - h264_cuvid
#        - -avoid_negative_ts
#        - make_zero
#        - -fflags
#        - +genpts
#        - -use_wallclock_as_timestamps
#        - '1'
 #       - -rtsp_transport
 #       - tcp

#      output_args:
#        clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
#        rtmp:  -c copy -f flv 

#      inputs:
#        - path: rtsp://192.168.178.55:8554/unicast
#          roles:
#            #- record
#            - detect
#            - rtmp
#            - clips
#    width: 1280
#    height: 720
#    fps: 5
#    clips:
#      enabled: True
#      pre_capture: 10
#      post_capture: 20
#    snapshots:
#      enabled: True
#      timestamp: True
#      bounding_box: True
#      crop: True

detectors:
  coral:
    type: edgetpu
    device: usb
#  cpu:
#   type: cpu

#ffmpeg:
#  input_args:
#    - -avoid_negative_ts
#    - make_zero
#    - -flags
#    - low_delay
#    - -strict
#    - experimental
#    - -fflags
#    - +genpts+discardcorrupt
#    - -rtsp_transport
#    - tcp
#    - -stimeout
#    - "5000000"
#    - -use_wallclock_as_timestamps
#    - "1"
#  output_args:
#
#    # Optional: output args for detect streams (default: shown below)
#    detect: -f rawvideo -pix_fmt yuv420p
#    # Optional: output args for record streams (default: shown below)
#    record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
#    # Optional: output args for clips streams (default: shown below)
#
#    # Optional: output args for rtmp streams (default: shown below)
#    rtmp: -c copy -f flv
#
#
#  hwaccel_args:
#    - -c:v
#    - h264_cuvid
timestamp_style:
  # Optional: Position of the timestamp (default: shown below)
  #           "tl" (top left), "tr" (top right), "bl" (bottom left), "br" (bottom right)
  position: "tl"
  # Optional: Format specifier conform to the Python package "datetime" (default: shown below)
  #           Additional Examples:
  #             german: "%d.%m.%Y %H:%M:%S"
  format: "%d/%m/%Y %H:%M:%S"
  # Optional: Color of font
  color:
    # All Required when color is specified (default: shown below)
    red: 255
    green: 255
    blue: 255
  # Optional: Line thickness of font (default: shown below)
  thickness: 2
  # Optional: Effect of lettering (default: shown below)
  #           None (No effect),
  #           "solid" (solid background in inverse color of font)
  #           "shadow" (shadow for font)

objects:

  track:
    - car
  #  - boat
    - person
    - bicycle
#    - motorcycle
    - bus
#    - train
    - bird
    - cat
    - dog
    - backpack
    - umbrella
#    - handbag
#    - suitcase
#    - sports ball
    - skateboard
#    - bottle
#    - wine glass
#    - cup
#    - fork
#    - knife
#    - spoon
#    - chair
#    - couch
#    - bed
#    - dining table
#    - toilet
#    - tv
#$    - laptop
#    - mouse
#    - remote
#    - keyboard
#    - cell phone
#    - microwave
#    - oven
#    - toaster
#    - sink
#    - refrigerator
#    - book
#    - clock
#    - vase
#    - scissors
#    - teddy bear
##    - hair drier
#    - toothbrush  
  filters:
    all:
      # Optional: minimum score for the object to initiate tracking (default: shown below)
      min_score: 0.5
      # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
      threshold: 0.7
      # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
      # Checks based on the bottom center of the bounding box of the object
birdseye:
  # Optional: Enable birdseye view (default: shown below)
  enabled: True
  # Optional: Width of the output resolution (default: shown below)
  width: 2560
  # Optional: Height of the output resolution (default: shown below)
  height: 1440
  # Optional: Encoding quality of the mpeg1 feed (default: shown below)
  # 1 is the highest quality, and 31 is the lowest. Lower quality feeds utilize less CPU resources.
  quality: 8
  # Optional: Mode of the view. Available options are: objects, motion, and continuous
  #   objects - cameras are included if they have had a tracked object within the last 30 seconds
  #   motion - cameras are included if motion was detected in the last 30 seconds
  #   continuous - all cameras are included always
  mode: motion

record:
  enabled: true
  retain_days: 0
  events:
    pre_capture: 10
    post_capture: 10
    retain:
      default: 3
    max_seconds: 300

motion:
  # Optional: The threshold passed to cv2.threshold to determine if a pixel is different enough to be counted as motion. (default: shown below)
  # Increasing this value will make motion detection less sensitive and decreasing it will make motion detection more sensitive.
  # The value should be between 1 and 255.
  threshold: 25
  # Optional: Minimum size in pixels in the resized motion image that counts as motion
  # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will make motion detection more sensitive to smaller
  # moving objects.
  contour_area: 100
  # Optional: Alpha value passed to cv2.accumulateWeighted when averaging the motion delta across multiple frames (default: shown below)
  # Higher values mean the current frame impacts the delta a lot, and a single raindrop may register as motion.
  # Too low and a fast moving person wont be detected as motion.
  delta_alpha: 0.2
  # Optional: Alpha value passed to cv2.accumulateWeighted when averaging frames to determine the background (default: shown below)
  # Higher values mean the current frame impacts the average a lot, and a new object will be averaged into the background faster.
  # Low values will cause things like moving shadows to be detected as motion for longer.
  # https://www.geeksforgeeks.org/background-subtraction-in-an-image-using-concept-of-running-average/
  frame_alpha: 0.2
  # Optional: Height of the resized motion frame  (default: 1/6th of the original frame height)
  # This operates as an efficient blur alternative. Higher values will result in more granular motion detection at the expense of higher CPU usage.
  # Lower values result in less CPU, but small changes may not register as motion.
  frame_height: 180
detect:
  max_disappeared: 100
  # Optional: Frequency for running detection on stationary objects (default: 10x the frame rate)
#  stationary_interval: 50
  fps: 5

Relevant log output

none

FFprobe output from your camera

n/a

Frigate stats

{
back: {
camera_fps: 5.1,
capture_pid: 242,
detection_fps: 0.7,
pid: 239,
process_fps: 5.1,
skipped_fps: 0
},
detection_fps: 0.7,
detectors: {
coral: {
detection_start: 0,
inference_speed: 10.02,
pid: 599
}
},
front: {
camera_fps: 4.8,
capture_pid: 247,
detection_fps: 0,
pid: 241,
process_fps: 4.8,
skipped_fps: 0
},
service: {
storage: {
/dev/shm: {
free: 3194.4,
mount_type: "tmpfs",
total: 3221.2,
used: 26.8
},
/media/frigate/clips: {
free: 89598.4,
mount_type: "ext4",
total: 117612,
used: 21995.2
},
/media/frigate/recordings: {
free: 89598.4,
mount_type: "ext4",
total: 117612,
used: 21995.2
},
/tmp/cache: {
free: 89598.4,
mount_type: "ext4",
total: 117612,
used: 21995.2
}
},
uptime: 295,
version: "0.9.4-26ae608"
}
}

Operating system

Other Linux

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

Reolink 510W and 410W

Any other information that may be helpful

No response

NickM-27 commented 2 years ago

Can you put the FFprobe or the camera info from VLC? Based on what I can see in the docs, you are using the hardware accelerator for H.265, are your cameras outputting H.265?

Sloth-on-meth commented 2 years ago

as far as i know i am using H264_CUVID, which is H264, right? i use the recommended config from the docs

https://docs.frigate.video/configuration/camera_specific/#reolink-410520-possibly-others

Sloth-on-meth commented 2 years ago

will post FFprobe when i can

NickM-27 commented 2 years ago

as far as i know i am using H264_CUVID, which is H264, right? i use the recommended config from the docs

https://docs.frigate.video/configuration/camera_specific/#reolink-410520-possibly-others

Oops, sorry misread, yeah that is H264. Are you able to tell if your GPU is being utilized or is it idle?

Sloth-on-meth commented 2 years ago

Oops, sorry misread, yeah that is H264. Are you able to tell if your GPU is being utilized or is it idle?

i can tell by running nvidia-smi that it is being used by ffmpeg - which makes it even stranger. is it perhaps only decoding and not encoding? or something like that?

besides, I'm having some more issues with frigate currently, see https://github.com/blakeblackshear/frigate/issues/2783 so my issue might be deeper

NickM-27 commented 2 years ago

That is odd, it does seem that the instruction for NVIDIA GPU is just being used for decoding so that is possible. In any case I wouldn't expect a CPU to hit 100%. Perhaps your other issue is causing some weirdness for this as well.

blakeblackshear commented 2 years ago

What is the frame rate coming from the camera?

Does this only happen when viewing the video in the UI? You can try removing your custom settings for live. It's going to take a good amount of power to encode the live stream at that resolution and frame rate. Note that live always uses the detect stream, so you are upscaling the low resolution stream on one camera.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.