search

bairhys / prometheus-frigate-exporter

A Prometheus exporter for Frigate stats
MIT License
33 stars 4 forks source link

No data in Grafana, minimal in Frigate Exporter metrics #13

Closed DaveTSG closed 1 month ago

DaveTSG commented 1 month ago

Hello,

I have just installed Prometheus-Frigate-Exporter and am having some difficulty getting the data into Grafana.

I already have Frigate and Prometheus running in pre-existing Docker containers. Both are in the "proxy" Docker network (a bridge network). I created a new docker-compose file for Prometheus-Frigate-Exporter, specifically, which contains:

services:
  frigate-exporter:
    image: rhysbailey/prometheus-frigate-exporter:latest
    container_name: frigate-exporter
    read_only: true
    restart: always
    networks:
      - proxy
    environment:
      FRIGATE_STATS_URL: http://frigate:5000/api/stats
    ports:
      - 9100:9100

networks:
  proxy:
    external: true

Loading https://frigate.davesservers.com/api/stats in my browser shows quite a bit of data. Ditto loading the PFE metrics page http://192.168.4.7:9100/metrics in my browser shows many lines of output, similar to what is shown on your page:

# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 187.0
python_gc_objects_collected_total{generation="1"} 71.0
python_gc_objects_collected_total{generation="2"} 0.0
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
# HELP python_gc_collections_total Number of times this generation was collected
# TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 35.0
python_gc_collections_total{generation="1"} 3.0
python_gc_collections_total{generation="2"} 0.0
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="12",patchlevel="3",version="3.12.3"} 1.0
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 3.3939456e+07
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 2.3973888e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.72188744824e+09
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.28
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 6.0
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1.048576e+06
# HELP frigate_audio_dBFS Audio dBFS for camera
# TYPE frigate_audio_dBFS gauge
frigate_audio_dBFS{camera_name="FrontDoor"} 0.0
frigate_audio_dBFS{camera_name="GarageCam"} 0.0
frigate_audio_dBFS{camera_name="PackageCam"} 0.0
# HELP frigate_audio_rms Audio RMS for camera
# TYPE frigate_audio_rms gauge
frigate_audio_rms{camera_name="FrontDoor"} 0.0
frigate_audio_rms{camera_name="GarageCam"} 0.0
frigate_audio_rms{camera_name="PackageCam"} 0.0
# HELP frigate_camera_fps Frames per second being consumed from your camera.
# TYPE frigate_camera_fps gauge
frigate_camera_fps{camera_name="FrontDoor"} 5.1
frigate_camera_fps{camera_name="GarageCam"} 5.1
frigate_camera_fps{camera_name="PackageCam"} 5.2
# HELP frigate_detection_enabled Detection enabled for camera
# TYPE frigate_detection_enabled gauge
frigate_detection_enabled{camera_name="FrontDoor"} 1.0
frigate_detection_enabled{camera_name="GarageCam"} 1.0
frigate_detection_enabled{camera_name="PackageCam"} 1.0
# HELP frigate_detection_fps Number of times detection is run per second.
# TYPE frigate_detection_fps gauge
frigate_detection_fps{camera_name="FrontDoor"} 11.7
frigate_detection_fps{camera_name="GarageCam"} 0.0
frigate_detection_fps{camera_name="PackageCam"} 5.2
# HELP frigate_process_fps Frames per second being processed by frigate.
# TYPE frigate_process_fps gauge
frigate_process_fps{camera_name="FrontDoor"} 5.1
frigate_process_fps{camera_name="GarageCam"} 5.1
frigate_process_fps{camera_name="PackageCam"} 5.2
# HELP frigate_skipped_fps Frames per second skip for processing by frigate.
# TYPE frigate_skipped_fps gauge
frigate_skipped_fps{camera_name="FrontDoor"} 0.0
frigate_skipped_fps{camera_name="GarageCam"} 0.0
frigate_skipped_fps{camera_name="PackageCam"} 0.0
# HELP frigate_bandwidth_usages_kBps bandwidth usages kilobytes per second
# TYPE frigate_bandwidth_usages_kBps gauge
frigate_bandwidth_usages_kBps{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/hR3g7Ce9h8UqzBLo?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/GarageCam@%Y%m%d%H%M%S%z.mp4",name="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/hR3g7Ce9h8UqzBLo?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/GarageCam@%Y%m%d%H%M%S%z.mp4",pid="4195",process="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/hR3g7Ce9h8UqzBLo?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/GarageCam@%Y%m%d%H%M%S%z.mp4"} 259.8
frigate_bandwidth_usages_kBps{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/NhxjoXQg5OVd0KiN?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac /tmp/cache/FrontDoor@%Y%m%d%H%M%S%z.mp4",name="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/NhxjoXQg5OVd0KiN?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac /tmp/cache/FrontDoor@%Y%m%d%H%M%S%z.mp4",pid="4196",process="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/NhxjoXQg5OVd0KiN?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac /tmp/cache/FrontDoor@%Y%m%d%H%M%S%z.mp4"} 98.5
frigate_bandwidth_usages_kBps{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/fLIVSyOaCypG4V7R?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=1280:h=1280:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",name="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/fLIVSyOaCypG4V7R?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=1280:h=1280:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",pid="4258",process="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/fLIVSyOaCypG4V7R?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=1280:h=1280:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"} 99.7
frigate_bandwidth_usages_kBps{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/HZ6BuVp4F3iWXAwl?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=960:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",name="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/HZ6BuVp4F3iWXAwl?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=960:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",pid="4260",process="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/HZ6BuVp4F3iWXAwl?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=960:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"} 48.3
frigate_bandwidth_usages_kBps{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/7EBFyiGKgiRhf0jU?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/PackageCam@%Y%m%d%H%M%S%z.mp4 -r 5 -vf fps=5,scale_vaapi=w=1600:h=1200:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",name="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/7EBFyiGKgiRhf0jU?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/PackageCam@%Y%m%d%H%M%S%z.mp4 -r 5 -vf fps=5,scale_vaapi=w=1600:h=1200:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",pid="4313",process="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/7EBFyiGKgiRhf0jU?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/PackageCam@%Y%m%d%H%M%S%z.mp4 -r 5 -vf fps=5,scale_vaapi=w=1600:h=1200:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"} 106.0
# HELP frigate_detection_total_fps Sum of detection_fps across all cameras and detectors.
# TYPE frigate_detection_total_fps gauge
frigate_detection_total_fps 16.9
# HELP frigate_detector_inference_speed_seconds Time spent running object detection in seconds.
# TYPE frigate_detector_inference_speed_seconds gauge
frigate_detector_inference_speed_seconds{name="coral"} 0.007690000000000001
# HELP frigate_detection_start Detector start time (unix timestamp)
# TYPE frigate_detection_start gauge
frigate_detection_start{name="coral"} 0.0
# HELP frigate_cpu_usage_percent Process CPU usage %
# TYPE frigate_cpu_usage_percent gauge
frigate_cpu_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/HZ6BuVp4F3iWXAwl?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=960:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",name="FrontDoor",pid="4260",process="ffmpeg",type="Camera"} 2.2
frigate_cpu_usage_percent{cmdline="frigate.capture:FrontDoor",name="FrontDoor",pid="408",process="capture",type="Camera"} 0.6
frigate_cpu_usage_percent{cmdline="frigate.process:FrontDoor",name="FrontDoor",pid="403",process="detect",type="Camera"} 5.7
frigate_cpu_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/fLIVSyOaCypG4V7R?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=1280:h=1280:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",name="GarageCam",pid="4258",process="ffmpeg",type="Camera"} 4.2
frigate_cpu_usage_percent{cmdline="frigate.capture:GarageCam",name="GarageCam",pid="416",process="capture",type="Camera"} 1.2
frigate_cpu_usage_percent{cmdline="frigate.process:GarageCam",name="GarageCam",pid="406",process="detect",type="Camera"} 0.6
frigate_cpu_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/7EBFyiGKgiRhf0jU?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/PackageCam@%Y%m%d%H%M%S%z.mp4 -r 5 -vf fps=5,scale_vaapi=w=1600:h=1200:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",name="PackageCam",pid="4313",process="ffmpeg",type="Camera"} 7.6
frigate_cpu_usage_percent{cmdline="frigate.capture:PackageCam",name="PackageCam",pid="412",process="capture",type="Camera"} 1.5
frigate_cpu_usage_percent{cmdline="frigate.process:PackageCam",name="PackageCam",pid="404",process="detect",type="Camera"} 3.0
frigate_cpu_usage_percent{cmdline="frigate.detector.coral",name="coral",pid="381",process="coral",type="detectors"} 4.8
frigate_cpu_usage_percent{cmdline="/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml",name="go2rtc",pid="90",process="go2rtc",type="go2rtc"} 0.0
frigate_cpu_usage_percent{cmdline="frigate.logger       ",name="logger",pid="369",process="logger",type="logger"} 0.0
frigate_cpu_usage_percent{cmdline="frigate.recording_manager",name="recording",pid="371",process="recording",type="recording"} 0.8
frigate_cpu_usage_percent{cmdline="/package/admin/s6/command/s6-svscan -d4 -- /run/service",name="/package/admin/s6/command/s6-svscan -d4 -- /run/service",pid="1",process="/package/admin/s6/command/s6-svscan -d4 -- /run/service",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise s6-linux-init-shutdownd",name="s6-supervise s6-linux-init-shutdownd",pid="16",process="s6-supervise s6-linux-init-shutdownd",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B",name="/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B",pid="17",process="/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise nginx",name="s6-supervise nginx",pid="25",process="s6-supervise nginx",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise s6rc-fdholder",name="s6-supervise s6rc-fdholder",pid="26",process="s6-supervise s6rc-fdholder",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise s6rc-oneshot-runner",name="s6-supervise s6rc-oneshot-runner",pid="27",process="s6-supervise s6rc-oneshot-runner",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise go2rtc-healthcheck",name="s6-supervise go2rtc-healthcheck",pid="28",process="s6-supervise go2rtc-healthcheck",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise go2rtc-log",name="s6-supervise go2rtc-log",pid="29",process="s6-supervise go2rtc-log",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise frigate-log",name="s6-supervise frigate-log",pid="30",process="s6-supervise frigate-log",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise go2rtc",name="s6-supervise go2rtc",pid="31",process="s6-supervise go2rtc",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise nginx-log",name="s6-supervise nginx-log",pid="32",process="s6-supervise nginx-log",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-supervise frigate",name="s6-supervise frigate",pid="33",process="s6-supervise frigate",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules",name="/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules",pid="42",process="/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --",name="/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --",pid="43",process="/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx",name="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx",pid="81",process="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc",name="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc",pid="82",process="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate",name="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate",pid="83",process="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="bash ./run.user go2rtc-healthcheck",name="bash ./run.user go2rtc-healthcheck",pid="98",process="bash ./run.user go2rtc-healthcheck",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="python3 -u -m frigate",name="python3 -u -m frigate",pid="99",process="python3 -u -m frigate",type="Other"} 1.0
frigate_cpu_usage_percent{cmdline="nginx: master process nginx",name="nginx: master process nginx",pid="115",process="nginx: master process nginx",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="145",process="nginx: worker process",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="146",process="nginx: worker process",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="158",process="nginx: worker process",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="175",process="nginx: worker process",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="192",process="nginx: worker process",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="209",process="nginx: worker process",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="nginx: cache manager process",name="nginx: cache manager process",pid="216",process="nginx: cache manager process",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(49)",name="/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(49)",pid="380",process="/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(49)",type="Other"} 0.1
frigate_cpu_usage_percent{cmdline="frigate.output       ",name="frigate.output       ",pid="383",process="frigate.output       ",type="Other"} 0.6
frigate_cpu_usage_percent{cmdline="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 960x720 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",name="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 960x720 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",pid="386",process="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 960x720 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1600x1200 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",name="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1600x1200 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",pid="387",process="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1600x1200 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x1280 -i pipe: -f mpegts -s 720x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",name="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x1280 -i pipe: -f mpegts -s 720x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",pid="388",process="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x1280 -i pipe: -f mpegts -s 720x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",name="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",pid="392",process="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",type="Other"} 0.0
frigate_cpu_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/hR3g7Ce9h8UqzBLo?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/GarageCam@%Y%m%d%H%M%S%z.mp4",name="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/hR3g7Ce9h8UqzBLo?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/GarageCam@%Y%m%d%H%M%S%z.mp4",pid="4195",process="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/hR3g7Ce9h8UqzBLo?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/GarageCam@%Y%m%d%H%M%S%z.mp4",type="Other"} 2.4
frigate_cpu_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/NhxjoXQg5OVd0KiN?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac /tmp/cache/FrontDoor@%Y%m%d%H%M%S%z.mp4",name="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/NhxjoXQg5OVd0KiN?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac /tmp/cache/FrontDoor@%Y%m%d%H%M%S%z.mp4",pid="4196",process="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/NhxjoXQg5OVd0KiN?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac /tmp/cache/FrontDoor@%Y%m%d%H%M%S%z.mp4",type="Other"} 3.2
frigate_cpu_usage_percent{cmdline="sleep 30s",name="sleep 30s",pid="15572",process="sleep 30s",type="Other"} 0.0
# HELP frigate_mem_usage_percent Process memory usage %
# TYPE frigate_mem_usage_percent gauge
frigate_mem_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/HZ6BuVp4F3iWXAwl?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=960:h=720:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",name="FrontDoor",pid="4260",process="ffmpeg",type="Camera"} 0.4
frigate_mem_usage_percent{cmdline="frigate.capture:FrontDoor",name="FrontDoor",pid="408",process="capture",type="Camera"} 0.8
frigate_mem_usage_percent{cmdline="frigate.process:FrontDoor",name="FrontDoor",pid="403",process="detect",type="Camera"} 0.9
frigate_mem_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/fLIVSyOaCypG4V7R?enableSrtp?video&audio -r 5 -vf fps=5,scale_vaapi=w=1280:h=1280:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",name="GarageCam",pid="4258",process="ffmpeg",type="Camera"} 0.5
frigate_mem_usage_percent{cmdline="frigate.capture:GarageCam",name="GarageCam",pid="416",process="capture",type="Camera"} 0.8
frigate_mem_usage_percent{cmdline="frigate.process:GarageCam",name="GarageCam",pid="406",process="detect",type="Camera"} 1.0
frigate_mem_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/7EBFyiGKgiRhf0jU?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/PackageCam@%Y%m%d%H%M%S%z.mp4 -r 5 -vf fps=5,scale_vaapi=w=1600:h=1200:format=nv12,hwdownload,format=nv12,format=yuv420p -threads 2 -f rawvideo -pix_fmt yuv420p pipe:",name="PackageCam",pid="4313",process="ffmpeg",type="Camera"} 0.5
frigate_mem_usage_percent{cmdline="frigate.capture:PackageCam",name="PackageCam",pid="412",process="capture",type="Camera"} 0.8
frigate_mem_usage_percent{cmdline="frigate.process:PackageCam",name="PackageCam",pid="404",process="detect",type="Camera"} 1.1
frigate_mem_usage_percent{cmdline="frigate.detector.coral",name="coral",pid="381",process="coral",type="detectors"} 0.9
frigate_mem_usage_percent{cmdline="/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml",name="go2rtc",pid="90",process="go2rtc",type="go2rtc"} 0.1
frigate_mem_usage_percent{cmdline="frigate.logger       ",name="logger",pid="369",process="logger",type="logger"} 0.7
frigate_mem_usage_percent{cmdline="frigate.recording_manager",name="recording",pid="371",process="recording",type="recording"} 0.9
frigate_mem_usage_percent{cmdline="/package/admin/s6/command/s6-svscan -d4 -- /run/service",name="/package/admin/s6/command/s6-svscan -d4 -- /run/service",pid="1",process="/package/admin/s6/command/s6-svscan -d4 -- /run/service",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise s6-linux-init-shutdownd",name="s6-supervise s6-linux-init-shutdownd",pid="16",process="s6-supervise s6-linux-init-shutdownd",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B",name="/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B",pid="17",process="/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise nginx",name="s6-supervise nginx",pid="25",process="s6-supervise nginx",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise s6rc-fdholder",name="s6-supervise s6rc-fdholder",pid="26",process="s6-supervise s6rc-fdholder",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise s6rc-oneshot-runner",name="s6-supervise s6rc-oneshot-runner",pid="27",process="s6-supervise s6rc-oneshot-runner",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise go2rtc-healthcheck",name="s6-supervise go2rtc-healthcheck",pid="28",process="s6-supervise go2rtc-healthcheck",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise go2rtc-log",name="s6-supervise go2rtc-log",pid="29",process="s6-supervise go2rtc-log",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise frigate-log",name="s6-supervise frigate-log",pid="30",process="s6-supervise frigate-log",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise go2rtc",name="s6-supervise go2rtc",pid="31",process="s6-supervise go2rtc",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise nginx-log",name="s6-supervise nginx-log",pid="32",process="s6-supervise nginx-log",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-supervise frigate",name="s6-supervise frigate",pid="33",process="s6-supervise frigate",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules",name="/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules",pid="42",process="/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --",name="/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --",pid="43",process="/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx",name="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx",pid="81",process="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc",name="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc",pid="82",process="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate",name="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate",pid="83",process="s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="bash ./run.user go2rtc-healthcheck",name="bash ./run.user go2rtc-healthcheck",pid="98",process="bash ./run.user go2rtc-healthcheck",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="python3 -u -m frigate",name="python3 -u -m frigate",pid="99",process="python3 -u -m frigate",type="Other"} 1.7
frigate_mem_usage_percent{cmdline="nginx: master process nginx",name="nginx: master process nginx",pid="115",process="nginx: master process nginx",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="145",process="nginx: worker process",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="146",process="nginx: worker process",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="158",process="nginx: worker process",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="175",process="nginx: worker process",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="192",process="nginx: worker process",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="nginx: worker process",name="nginx: worker process",pid="209",process="nginx: worker process",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="nginx: cache manager process",name="nginx: cache manager process",pid="216",process="nginx: cache manager process",type="Other"} 0.0
frigate_mem_usage_percent{cmdline="/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(49)",name="/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(49)",pid="380",process="/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(49)",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="frigate.output       ",name="frigate.output       ",pid="383",process="frigate.output       ",type="Other"} 0.8
frigate_mem_usage_percent{cmdline="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 960x720 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",name="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 960x720 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",pid="386",process="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 960x720 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1600x1200 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",name="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1600x1200 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",pid="387",process="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1600x1200 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x1280 -i pipe: -f mpegts -s 720x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",name="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x1280 -i pipe: -f mpegts -s 720x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",pid="388",process="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x1280 -i pipe: -f mpegts -s 720x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",type="Other"} 0.1
frigate_mem_usage_percent{cmdline="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",name="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",pid="392",process="ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:",type="Other"} 0.4
frigate_mem_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/hR3g7Ce9h8UqzBLo?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/GarageCam@%Y%m%d%H%M%S%z.mp4",name="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/hR3g7Ce9h8UqzBLo?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/GarageCam@%Y%m%d%H%M%S%z.mp4",pid="4195",process="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/hR3g7Ce9h8UqzBLo?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/GarageCam@%Y%m%d%H%M%S%z.mp4",type="Other"} 0.2
frigate_mem_usage_percent{cmdline="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/NhxjoXQg5OVd0KiN?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac /tmp/cache/FrontDoor@%Y%m%d%H%M%S%z.mp4",name="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/NhxjoXQg5OVd0KiN?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac /tmp/cache/FrontDoor@%Y%m%d%H%M%S%z.mp4",pid="4196",process="ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.13.2-6476f8a -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsps://192.168.1.1:7441/NhxjoXQg5OVd0KiN?enableSrtp?video&audio -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac /tmp/cache/FrontDoor@%Y%m%d%H%M%S%z.mp4",type="Other"} 0.2
frigate_mem_usage_percent{cmdline="sleep 30s",name="sleep 30s",pid="15572",process="sleep 30s",type="Other"} 0.0
# HELP frigate_gpu_usage_percent GPU utilisation %
# TYPE frigate_gpu_usage_percent gauge
frigate_gpu_usage_percent{gpu_name="intel-vaapi"} 3.28
# HELP frigate_gpu_mem_usage_percent GPU memory usage %
# TYPE frigate_gpu_mem_usage_percent gauge
# HELP frigate_service_info Frigate version info
# TYPE frigate_service_info gauge
frigate_service_info{latest_version="0.13.2",version="0.13.2-6476f8a"} 1.0
# HELP frigate_service_uptime_seconds Uptime seconds
# TYPE frigate_service_uptime_seconds gauge
frigate_service_uptime_seconds 5363.0
# HELP frigate_service_last_updated_timestamp Stats recorded time (unix timestamp)
# TYPE frigate_service_last_updated_timestamp gauge
frigate_service_last_updated_timestamp 1.721887864e+09
# HELP frigate_device_temperature Device Temperature
# TYPE frigate_device_temperature gauge
# HELP frigate_storage_free_bytes Storage free bytes
# TYPE frigate_storage_free_bytes gauge
frigate_storage_free_bytes{storage="/dev/shm"} 1.168e+08
frigate_storage_free_bytes{storage="/media/frigate/clips"} 3.93893903e+013
frigate_storage_free_bytes{storage="/media/frigate/recordings"} 3.93893903e+013
frigate_storage_free_bytes{storage="/tmp/cache"} 9.454e+08
# HELP frigate_storage_mount_type_info Storage mount type
# TYPE frigate_storage_mount_type_info gauge
frigate_storage_mount_type_info{mount_type="tmpfs",storage="/dev/shm"} 1.0
frigate_storage_mount_type_info{mount_type="cifs",storage="/media/frigate/clips"} 1.0
frigate_storage_mount_type_info{mount_type="cifs",storage="/media/frigate/recordings"} 1.0
frigate_storage_mount_type_info{mount_type="tmpfs",storage="/tmp/cache"} 1.0
# HELP frigate_storage_total_bytes Storage total bytes
# TYPE frigate_storage_total_bytes gauge
frigate_storage_total_bytes{storage="/dev/shm"} 1.28e+08
frigate_storage_total_bytes{storage="/media/frigate/clips"} 6.40656998e+013
frigate_storage_total_bytes{storage="/media/frigate/recordings"} 6.40656998e+013
frigate_storage_total_bytes{storage="/tmp/cache"} 9.537e+08
# HELP frigate_storage_used_bytes Storage used bytes
# TYPE frigate_storage_used_bytes gauge
frigate_storage_used_bytes{storage="/dev/shm"} 1.12e+07
frigate_storage_used_bytes{storage="/media/frigate/clips"} 2.46763095e+013
frigate_storage_used_bytes{storage="/media/frigate/recordings"} 2.46763095e+013
frigate_storage_used_bytes{storage="/tmp/cache"} 8.300000000000001e+06
# HELP frigate_camera_events_total Count of camera events since exporter started
# TYPE frigate_camera_events_total counter
frigate_camera_events_total{camera="FrontDoor",label="car"} 1.0

If I load Prometheus targets list (at http://192.168.4.7:9090/targets), it shows my pre-existing target (Crowdsec), which I configured when I set up Prometheus (earlier today). The Crowdsec target is up and running fine, and the data is being shown in Grafana.

instance="192.168.4.7:9100" job="prometheus_frigate_exporter" Discovered labels: address="192.168.4.7:9100" metrics_path="/metrics" scheme="http" scrape_interval="15s" __scrape_timeout__="10s" job="prometheus_frigate_exporter"

My prometheus.yml file contains the following:

# my global config
global:
  scrape_interval: 15s
  evaluation_interval: 15s
  body_size_limit: 15MB
  sample_limit: 1500
  target_limit: 30
  label_limit: 30
  label_name_length_limit: 200
  label_value_length_limit: 200
  # scrape_timeout is set to the global default (10s).

scrape_configs:
- job_name: crowdsec_myMachine
  static_configs:
    - targets: ["192.168.4.7:6060"]
- job_name: prometheus
  static_configs:
    - targets: ["localhost:9090"]
- job_name: prometheus_frigate_exporter
  static_configs:
    - targets: ["192.168.4.7:9100"]

The Prometheus-Frigate-Exporter logs show all ok:

2024-07-25T06:04:09.124187874Z 2024-07-25 06:04:09,123 - Started, Frigate API URL: http://frigate:5000/api/stats
2024-07-25T06:04:09.124217954Z 2024-07-25 06:04:09,124 - Metrics at: http://localhost:9100/metrics

I am not sure if it is relevant or not, but I also have Traefik (proxy and load balancer) installed. It is not being used with Prometheus-Frigate-Exporter, but it is working with Frigate itself. Here's my Frigate docker-compose.yml file:

services:
  frigate:
    container_name: frigate
    restart: unless-stopped
    env_file: .env
    image: ghcr.io/blakeblackshear/frigate:stable
    cap_add:
      - CAP_PERFMON
    shm_size: "128mb"
    devices:
      - /dev/bus/usb:/dev/bus/usb  # Enables Google Coral EdgeAI USB TPU
      - /dev/dri/renderD128:/dev/dri/renderD128 # Intel-based iGPU hardware acceleration.
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/david/dockerfilesNotOnNAS/frigate-config:/config
      - ./storage:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8555:8555"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.frigate.rule=Host(`frigate.davesservers.com`)"
      - "traefik.http.routers.frigate.entrypoints=https"
      - "traefik.http.routers.frigate.tls=true"
      - "traefik.http.services.frigate.loadbalancer.server.port=5000"
      - homepage.group=Home admin
      - homepage.name=Frigate
      - homepage.icon=frigate
      - homepage.href=https://frigate.davesservers.com/birdseye
      - homepage.description=Security cameras
      - homepage.statusStyle=dot
      - homepage.showStats=true
      - homepage.widget.type=frigate
      - homepage.widget.url=https://frigate.davesservers.com
      - homepage.widget.enableRecentEvents=true
    networks:
      - proxy

networks:
  proxy:
    external: true

I was able to create a new data source in Grafana, along with the dashboard as per the instructions, but the dashboard itself just shows a bunch of "no data", which makes sense, given the Prometheus target is "down".

Would anyone know why this might be and how to fix the issue please?

Thank you.

bairhys commented 1 month ago

Hey @DaveTSG,

Looks like your Prometheus setup doesn't like the length of the labels that this Frigate Prometheus exporter outputs from what you shown here:

Error: label_value_length_limit exceeded (metric: frigate_bandwidth_usages_kBps, label name: cmdline, value: "ffmpeg -hide_banner -loglevel warning -threads 2 -", length: 543, limit: 200)

Frigate commands are stored in this exporters metric labels and Frigate commands can get really long. The commands are shown in Grafana to provide some context when looking at process CPU and MEM stats. For example, if you test different frame rates or resolutions, can look at the command to see what settings were used.

Googling label_value_length_limit finds the Prometheus configuration page and one of the settings is this:

# Per-scrape limit on length of labels value that will be accepted for a sample.
  # If a label value is longer than this number post metric-relabeling, the
  # entire scrape will be treated as failed. 0 means no limit.
  [ label_value_length_limit: <int> | default = 0 ]

Looks like your current label_value_length_limit is 200 and it failed the entire scrape because it found a label with length 543. So I think best solution would be to set your Prometheus label_value_length_limit: 0

DaveTSG commented 1 month ago

I think best solution would be to set your Prometheus label_value_length_limit: 0

Bingo! Working perfectly now. Thank you for your help!