blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.22k stars 1.76k forks source link

Unable to rotate camera image for portrait view #1242

Closed truxntrax closed 3 years ago

truxntrax commented 3 years ago

Describe the bug I have used the following output_arg to rotate the image of one of my cams, but I get error messages and a green screen for my camera. I have also tried inverting (swapping height and width 1080*1920)

rtmp: -vf transpose=2 -f -rawvideo -pix_fmt yuv420p and have tried:

rtmp: -vf transpose=1 -r 5 -f rawvideo -pix_fmt yuv420p

Version of frigate 0.8.4-5043040

Config file

mqtt:
  host: 192.168.1.101
  port: 1883
  user: ereacted
  password: redacted
cameras:
  back_facing_house:
    ffmpeg:
      inputs:
        - path: rtsp://viewer:redacted@192.168.1.70:555/user=admin_password=_channel=1_stream=0
          roles:
            - detect
            - rtmp
            - clips
    width: 1280
    height: 720
    fps: 5
  fri_side_door:
    ffmpeg:
      inputs:
        - path: rtsp://viewer:redacted@192.168.1.74:554/user=admin_password=_channel=1_stream=0
          roles:
            - detect
            - rtmp
            - clips
      output_args:
#        record: -vf transpose=1 -f rawvideo -pix_fmt -yuv420p
#        clips: -vf transpose=1 -f -rawvideo -pix_fmt yuv420p
        rtmp: -vf transpose=2 -f -rawvideo -pix_fmt yuv420p
    width: 1920
    height: 1080
    fps: 5
detectors:
  coral:
    type: edgetpu
    device: usb
objects:
  track:
    - person
    - dog
    - car

Frigate container logs

frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg process is not running. exiting capture thread...
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [NULL @ 0x5629e49fd540] Requested output format '-rawvideo' is not a suitable output format
frigate    | ffmpeg.fri_side_door.detect    ERROR   : rtmp://127.0.0.1/live/fri_side_door: Invalid argument
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg process is not running. exiting capture thread...
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [NULL @ 0x558547ad1fc0] Requested output format '-rawvideo' is not a suitable output format
frigate    | ffmpeg.fri_side_door.detect    ERROR   : rtmp://127.0.0.1/live/fri_side_door: Invalid argument
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg process is not running. exiting capture thread...
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [NULL @ 0x563a1e788fc0] Requested output format '-rawvideo' is not a suitable output format
frigate    | ffmpeg.fri_side_door.detect    ERROR   : rtmp://127.0.0.1/live/fri_side_door: Invalid argument
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg process is not running. exiting capture thread...
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [NULL @ 0x564f36f77540] Requested output format '-rawvideo' is not a suitable output format
frigate    | ffmpeg.fri_side_door.detect    ERROR   : rtmp://127.0.0.1/live/fri_side_door: Invalid argument
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg process is not running. exiting capture thread...
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [NULL @ 0x55a26e8b5440] Requested output format '-rawvideo' is not a suitable output format
frigate    | ffmpeg.fri_side_door.detect    ERROR   : rtmp://127.0.0.1/live/fri_side_door: Invalid argument
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg process is not running. exiting capture thread...

Frigate stats

  "back_facing_house": {
    "camera_fps": 5.0, 
    "capture_pid": 44, 
    "detection_fps": 0.0, 
    "pid": 42, 
    "process_fps": 5.0, 
    "skipped_fps": 0.0
  }, 
  "detection_fps": 0.0, 
  "detectors": {
    "coral": {
      "detection_start": 0.0, 
      "inference_speed": 9.33, 
      "pid": 39
    }
  }, 
  "fri_side_door": {
    "camera_fps": 0.0, 
    "capture_pid": 45, 
    "detection_fps": 0.0, 
    "pid": 43, 
    "process_fps": 0.0, 
    "skipped_fps": 0.0
  }, 
  "service": {
    "storage": {
      "/dev/shm": {
        "free": 65.0, 
        "mount_type": "tmpfs", 
        "total": 67.1, 
        "used": 2.1
      }, 
      "/media/frigate/clips": {
        "free": 274723.4, 
        "mount_type": "ext4", 
        "total": 4233172.6, 
        "used": 3776418.7
      }, 
      "/media/frigate/recordings": {
        "free": 274723.4, 
        "mount_type": "ext4", 
        "total": 4233172.6, 
        "used": 3776418.7
      }, 
      "/tmp/cache": {
        "free": 983.9, 
        "mount_type": "tmpfs", 
        "total": 1000.0, 
        "used": 16.1
      }
    }, 
    "uptime": 653, 
    "version": "0.8.4-5043040"
  }
}

FFprobe from your camera

Run the following command and paste output below

ffprobe <stream_url>
ffprobe rtsp://:@192.168.1.74:554/user=admin_password=_channel=1_stream=0
ffprobe version 3.4.4-1~16.04.york0 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version='1~16.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, rtsp, from 'rtsp://:@192.168.1.74:554/user=admin_password=_channel=1_stream=0':
  Metadata:
    title           : RTSP Session
  Duration: N/A, start: 0.200000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1920x1080, 5 fps, 5 tbr, 90k tbn, 180k tbc

Screenshots If applicable, add screenshots to help explain your problem.

Computer Hardware

Camera Info:

Additional context The camera works fine when I remove the output_args to attempt to rotate the image to accommodate portrait view. See screen shot. Screenshot 2021-06-16 21 19 32

blakeblackshear commented 3 years ago

It looks like you are using a modified version of the default detect output args for other roles. That's not going to work. You need to start with the default args for those specific roles and add to the transpose modifications to them.

truxntrax commented 3 years ago

Hi, Thanks for the reply are the default arg for each role in the docs? Or do I need to investigate for my individual installation?

Thanks

John

truxntrax commented 3 years ago

I think I am getting a little further... Sorry I didn't pickup the following I the docs:

output_args:
    # Optional: output args for detect streams (default: shown below)
    detect: -f rawvideo -pix_fmt yuv420p
    # Optional: output args for record streams (default: shown below)
    record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
    # Optional: output args for clips streams (default: shown below)
    clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
    # Optional: output args for rtmp streams (default: shown below)
    rtmp: -c copy -f flv

I now have the following in my config:

output_args:
        detect: -vf transpose=1 -f rawvideo -pix_fmt yuv420p
        record: -vf transpose=1 --f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
        clips: -vf transpose=1 -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
        rtmp: -vf transpose=1 -c copy -f flv

But I get the following error message:

frigate    | ffmpeg.fri_side_door.detect    ERROR   : Filtergraph 'transpose=1' was defined for video output stream 0:0 but codec copy was selected.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Filtering and streamcopy cannot be used together.
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg process is not running. exiting capture thread...

Any advise?

blakeblackshear commented 3 years ago

-c copy tells ffmpeg to copy the streams without modifying. Ffmpeg is complaining that you are trying to rotate the video, but also telling it to not re-encode. Try replacing -c copy with -c:v libx264. That tells ffmpeg to use h264 when encoding after rotation.

truxntrax commented 3 years ago

I'm seeing some great results now, thanks for your help. Screenshot 2021-06-18 13 26 42

I guess the trade-off for flipping the camera for the roles is that ffmpeg has to recode? I am now seeing that my most hungry process is ffmeg!:

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                            
24407 root      20   0 2531164 1.182g   3416 S 152.2  5.0 116:34.89 ffmpeg  

Could you help explain where the different roles are "seen"/ exposed? Perhaps I do not need to rotate for each role? I guess I could experiment by commenting out each in turn and observe the impact!

Is there anything I can do to address the CPU usage? Is this where hw_accelaration could help me?

blakeblackshear commented 3 years ago

There are ways to do hwaccel for encoding too. Rtmp is just to display in home assistant. Record is for 24/7 recordings. Clips is for videos of each tracked object.

truxntrax commented 3 years ago

Which is the role that displays the camera in the Frigate webpage, please?

Could you point me to any docs that help with where I configure hwaccel for encoding?

blakeblackshear commented 3 years ago

That would be detect, which is required.

Just search around for ffmpeg hwaccel arguments based on your host OS. You can also do hwaccel for decoding too. Ideally you want both. I would start by trying to get it working for decoding.

truxntrax commented 3 years ago

I will play with these settings later! Thanks! Great to have the image rotation. Thanks so much for your help. Zoneminder will be turned off soon - whoop! I will report back how I get on.

truxntrax commented 3 years ago

OK, I have tried what I think are the right hwaccel args for my hardware and see the following errors:

frigate    | ffmpeg.fri_side_door.detect    ERROR   : DRM_IOCTL_I915_GEM_APERTURE failed: Invalid argument
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Assuming 131072kB available aperture size.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : May lead to reduced performance or incorrect rendering.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : get chip id failed: -1 [22]
frigate    | ffmpeg.fri_side_door.detect    ERROR   : param: 4, val: 0
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [AVHWDeviceContext @ 0x55aa9ced7880] libva: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so init failed
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [AVHWDeviceContext @ 0x55aa9ced7880] Failed to initialise VAAPI connection: -1 (unknown libva error).
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Device creation failed: -5.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [h264 @ 0x55aa9cd04fc0] No device available for decoder: device type vaapi needed for codec h264.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate    | ffmpeg.back_facing_house.detect ERROR   : [rtsp @ 0x55b1d84faec0] Could not find codec parameters for stream 0 (Video: h264, none): unspecified size
frigate    | ffmpeg.back_facing_house.detect ERROR   : Consider increasing the value for the 'analyzeduration' and 'probesize' options
frigate    | ffmpeg.back_facing_house.detect ERROR   : Output file #0 does not contain any stream
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg process is not running. exiting capture thread...
frigate    | frigate.video                  INFO    : back_facing_house: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : back_facing_house: ffmpeg process is not running. exiting capture thread...
frigate    | ffmpeg.fri_side_door.detect    ERROR   : DRM_IOCTL_I915_GEM_APERTURE failed: Invalid argument
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Assuming 131072kB available aperture size.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : May lead to reduced performance or incorrect rendering.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : get chip id failed: -1 [22]
frigate    | ffmpeg.fri_side_door.detect    ERROR   : param: 4, val: 0
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [AVHWDeviceContext @ 0x55f20cecb4c0] libva: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so init failed
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [AVHWDeviceContext @ 0x55f20cecb4c0] Failed to initialise VAAPI connection: -1 (unknown libva error).
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Device creation failed: -5.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [h264 @ 0x55f20cec8fc0] No device available for decoder: device type vaapi needed for codec h264.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate    | ffmpeg.back_facing_house.detect ERROR   : [rtsp @ 0x55b6f0012ec0] Could not find codec parameters for stream 0 (Video: h264, none): unspecified size
frigate    | ffmpeg.back_facing_house.detect ERROR   : Consider increasing the value for the 'analyzeduration' and 'probesize' options
frigate    | ffmpeg.back_facing_house.detect ERROR   : Output file #0 does not contain any stream
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : fri_side_door: ffmpeg process is not running. exiting capture thread...
frigate    | frigate.video                  INFO    : back_facing_house: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
frigate    | frigate.video                  INFO    : back_facing_house: ffmpeg process is not running. exiting capture thread...
frigate    | ffmpeg.fri_side_door.detect    ERROR   : DRM_IOCTL_I915_GEM_APERTURE failed: Invalid argument
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Assuming 131072kB available aperture size.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : May lead to reduced performance or incorrect rendering.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : get chip id failed: -1 [22]
frigate    | ffmpeg.fri_side_door.detect    ERROR   : param: 4, val: 0
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [AVHWDeviceContext @ 0x55c7d53831c0] libva: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so init failed
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [AVHWDeviceContext @ 0x55c7d53831c0] Failed to initialise VAAPI connection: -1 (unknown libva error).
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Device creation failed: -5.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : [h264 @ 0x55c7d538cfc0] No device available for decoder: device type vaapi needed for codec h264.
frigate    | ffmpeg.fri_side_door.detect    ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate    | ffmpeg.back_facing_house.detect ERROR   : [rtsp @ 0x565131af3ec0] Could not find codec parameters for stream 0 (Video: h264, none): unspecified size
frigate    | ffmpeg.back_facing_house.detect ERROR   : Consider increasing the value for the 'analyzeduration' and 'probesize' options
frigate    | ffmpeg.back_facing_house.detect ERROR   : Output file #0 does not contain any stream

These are the values I added to my config file:

#ffmpeg:
#  hwaccel_args:
#   - -hwaccel
#    - vaapi
#    - -hwaccel_device
#    - /dev/dri/renderD128
#    - -hwaccel_output_format
#    - yuv420p`

And here are the details about my hardware:

H/W path                 Device       Class          Description
================================================================
                                      system         Precision WorkStation T5500
/0                                    bus            0CRH6C
/0/0                                  memory         64KiB BIOS
/0/400                                processor      Intel(R) Xeon(R) CPU           X5650  @ 2.67GHz
/0/400/700                            memory         384KiB L1 cache
/0/400/701                            memory         1536KiB L2 cache
/0/400/704                            memory         12MiB L3 cache
/0/401                                processor      Xeon
/0/401/702                            memory         384KiB L1 cache
/0/401/703                            memory         1536KiB L2 cache
/0/401/705                            memory         12MiB L3 cache
/0/1000                               memory         24GiB System Memory
/0/1000/0                             memory         4GiB DIMM DDR3 1333 MHz (0.8 ns)
/0/1000/1                             memory         4GiB DIMM DDR3 1333 MHz (0.8 ns)
/0/1000/2                             memory         4GiB DIMM DDR3 1333 MHz (0.8 ns)
/0/1000/3                             memory         4GiB DIMM DDR3 1333 MHz (0.8 ns)
/0/1000/4                             memory         4GiB DIMM DDR3 1333 MHz (0.8 ns)
/0/1000/5                             memory         4GiB DIMM DDR3 1333 MHz (0.8 ns)
/0/100                                bridge         5520 I/O Hub to ESI Port
/0/100/1                              bridge         5520/5500/X58 I/O Hub PCI Express Root Port 1
/0/100/1/0                            bridge         PCI Express to PCI-XPI7C9X130 PCI-X Bridge
/0/100/3                              bridge         5520/5500/X58 I/O Hub PCI Express Root Port 3
/0/100/3/0                            display        GF106GL [Quadro 2000]
/0/100/3/0.1                          multimedia     GF106 High Definition Audio Controller
/0/100/7                              bridge         5520/5500/X58 I/O Hub PCI Express Root Port 7
/0/100/7/0                            bus            VL805 USB 3.0 Host Controller
/0/100/7/0/0             usb10        bus            xHCI Host Controller
/0/100/7/0/0/1                        generic        Generic USB device
/0/100/7/0/1             usb9         bus            xHCI Host Controller
/0/100/7/0/1/1                        bus            USB2.0 Hub
/0/100/14                             generic        7500/5520/5500/X58 I/O Hub System Management Registers
/0/100/14.1                           generic        7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers
/0/100/14.2                           generic        7500/5520/5500/X58 I/O Hub Control Status and RAS Registers
/0/100/1a                             bus            82801JI (ICH10 Family) USB UHCI Controller #4
/0/100/1a/1              usb3         bus            UHCI Host Controller
/0/100/1a/1/2            scsi9        storage        Samsung M3 Portable
/0/100/1a/1/2/0.0.0      /dev/sdg     disk           1TB M3 Portable
/0/100/1a/1/2/0.0.0/1    /dev/sdg1    volume         931GiB HPFS/NTFS partition
/0/100/1a.1                           bus            82801JI (ICH10 Family) USB UHCI Controller #5
/0/100/1a.1/1            usb4         bus            UHCI Host Controller
/0/100/1a.1/1/2                       communication  Communication device
/0/100/1a.2                           bus            82801JI (ICH10 Family) USB UHCI Controller #6
/0/100/1a.2/1            usb5         bus            UHCI Host Controller
/0/100/1a.7                           bus            82801JI (ICH10 Family) USB2 EHCI Controller #2
/0/100/1a.7/1            usb1         bus            EHCI Host Controller
/0/100/1a.7/1/3          scsi6        storage        Elements 1078
/0/100/1a.7/1/3/0.0.0    /dev/sdb     disk           2TB Elements 1078
/0/100/1a.7/1/3/0.0.0/1  /dev/sdb1    volume         1862GiB HPFS/NTFS partition
/0/100/1b                             multimedia     82801JI (ICH10 Family) HD Audio Controller
/0/100/1c                             bridge         82801JI (ICH10 Family) PCI Express Root Port 1
/0/100/1c.5                           bridge         82801JI (ICH10 Family) PCI Express Root Port 6
/0/100/1c.5/0            enp6s0       network        NetXtreme BCM5761 Gigabit Ethernet PCIe
/0/100/1d                             bus            82801JI (ICH10 Family) USB UHCI Controller #1
/0/100/1d/1              usb6         bus            UHCI Host Controller
/0/100/1d.1                           bus            82801JI (ICH10 Family) USB UHCI Controller #2
/0/100/1d.1/1            usb7         bus            UHCI Host Controller
/0/100/1d.2                           bus            82801JI (ICH10 Family) USB UHCI Controller #3
/0/100/1d.2/1            usb8         bus            UHCI Host Controller
/0/100/1d.7                           bus            82801JI (ICH10 Family) USB2 EHCI Controller #1
/0/100/1d.7/1            usb2         bus            EHCI Host Controller
/0/100/1e                             bridge         82801 PCI Bridge
/0/100/1f                             bridge         82801JIR (ICH10R) LPC Interface Controller
/0/100/1f.2                           storage        SATA Controller [RAID mode]
/0/100/1f.3                           bus            82801JI (ICH10 Family) SMBus Controller
/0/7                                  bridge         5520/5500/X58 I/O Hub PCI Express Root Port 7
/0/9                                  bridge         7500/5520/5500/X58 I/O Hub PCI Express Root Port 9
/0/9/0                   scsi8        storage        SAS1068E PCI-Express Fusion-MPT SAS
/0/9/0/0.0.0             /dev/sdc     disk           300GB ST3300657SS
/0/9/0/0.0.0/1           /dev/sdc1    volume         100MiB Windows NTFS volume
/0/9/0/0.0.0/2           /dev/sdc2    volume         83GiB Windows NTFS volume
/0/9/0/0.0.0/3           /dev/sdc3    volume         450MiB Windows NTFS volume
/0/9/0/0.0.0/4           /dev/sdc4    volume         195GiB Extended partition
/0/9/0/0.0.0/4/5         /dev/sdc5    volume         171GiB Linux filesystem partition
/0/9/0/0.0.0/4/6         /dev/sdc6    volume         23GiB Linux swap / Solaris partition
/0/9/0/0.1.0             /dev/sdd     disk           500GB SAMSUNG HD501LJ
/0/9/0/0.1.0/1           /dev/sdd1    volume         465GiB EFI partition
/0/9/0/0.2.0             /dev/sde     disk           500GB SAMSUNG HD501LJ
/0/9/0/0.2.0/1           /dev/sde1    volume         465GiB EFI partition
/0/9/0/0.3.0             /dev/sdf     disk           300GB SAMSUNG HD300LJ
/0/9/0/0.3.0/1           /dev/sdf1    volume         279GiB EFI partition
/0/14                                 generic        7500/5520/5500/X58 I/O Hub System Management Registers
/0/14.1                               generic        7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers
/0/14.2                               generic        7500/5520/5500/X58 I/O Hub Control Status and RAS Registers
/0/101                                bridge         Xeon 5600 Series QuickPath Architecture Generic Non-core Registers
/0/102                                bridge         Xeon 5600 Series QuickPath Architecture System Address Decoder
/0/103                                bridge         Xeon 5600 Series QPI Link 0
/0/104                                bridge         Xeon 5600 Series QPI Physical 0
/0/105                                bridge         Xeon 5600 Series Mirror Port Link 0
/0/106                                bridge         Xeon 5600 Series Mirror Port Link 1
/0/107                                bridge         Xeon 5600 Series QPI Link 1
/0/108                                bridge         Xeon 5600 Series QPI Physical 1
/0/109                                bridge         Xeon 5600 Series Integrated Memory Controller Registers
/0/10a                                bridge         Xeon 5600 Series Integrated Memory Controller Target Address Decoder
/0/10b                                bridge         Xeon 5600 Series Integrated Memory Controller RAS Registers
/0/10c                                bridge         Xeon 5600 Series Integrated Memory Controller Test Registers
/0/10d                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 0 Control
/0/10e                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 0 Address
/0/10f                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 0 Rank
/0/110                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 0 Thermal Control
/0/111                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 1 Control
/0/112                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 1 Address
/0/113                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 1 Rank
/0/114                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 1 Thermal Control
/0/115                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 2 Control
/0/116                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 2 Address
/0/117                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 2 Rank
/0/118                                bridge         Xeon 5600 Series Integrated Memory Controller Channel 2 Thermal Control
/0/1                     scsi0        storage        
/0/1/0.0.0               /dev/cdrom   disk           DVD+-RW SH-216AB
/1                       vethc74d981  network        Ethernet interface

I appreciate all your guidance!

blakeblackshear commented 3 years ago

From what I can tell, the Xeon X5650 does not support QuickSync, so you don't have hardware acceleration on that CPU.

truxntrax commented 3 years ago

I think I may try to rotate my cams back to the standard orientation... It seems the power needed and CPU to process 2 of my cams to rotate 90 degrees side steps the graceful benefits of the lightweight nature of frigate.

blakeblackshear commented 3 years ago

Rotating for detect only shouldn't add much if any CPU load.

truxntrax commented 3 years ago

You are correct. That may be a reasonable compromise for my circumstances. It's a shame my camera firmware doesn't allow image rotation at source... I'm going to physically move one cam, but will be left with my front door cam rotated by 90 degrees. I mainly use this for a snapshot sent to me when someone rings the doorbell. I will need to figure out a way of rotating the jpg I pull from the camera stream in homeassistant. Hopefully, this is doable, as I view these a few times per day...

I learned a lot from your help - many thanks!

truxntrax commented 3 years ago

This is my video card details:

 *-display               
       description: VGA compatible controller
       product: GF106GL [Quadro 2000]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:03:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nouveau latency=0
       resources: irq:31 memory:f4000000-f5ffffff memory:e0000000-e7ffffff memory:e8000000-ebffffff ioport:cc80(size=128) memory:f7e00000-f7e7ffff

I am currently using this image: image: blakeblackshear/frigate:stable-amd64

Will I gain anything by swapping to the image that supports nvidia?:

blakeblackshear/frigate:stable-amd64nvidia

I guess I just insert this new image in my docker-compose config file and spin it up?

Thanks

blakeblackshear commented 3 years ago

The snapshots that frigate saves come from the detect stream, so they will be rotated.

blakeblackshear commented 3 years ago

Will I gain anything by swapping to the image that supports nvidia?

Not automatically. You need to determine if that card supports hwaccel in ffmpeg.

noelhibbard commented 3 years ago

@truxntrax what were your final working output args (detect) to get it to rotate? I'm not even concerned about hw encoding yet.

truxntrax commented 3 years ago

Here are what i used:

      output_args:
        detect: -vf transpose=1 -f rawvideo -pix_fmt yuv420p
noelhibbard commented 3 years ago

@truxntrax I tried a gazillion options without luck including these exact options. I finally gave up and rolled back my config to a backup. But now I just tried your options and this time it's working. I must have had something else in my config that was conflicting. Now I will see if I can get it to use hardware. Thanks for the quick response!

noelhibbard commented 3 years ago

So I got the detector rotated but I'm trying to rotate the other stuff without re-encoding. I can run one of the videos from my doorbell through ffmpeg like this and then it plays back perfect: ffmpeg -i 55.55.mp4 -map_metadata 0 -metadata:s:v rotate="-90" -codec copy 55.55r.mp4

But once I introduce -f segment it's like it ignores my request to add metadata. It's strange because -metadata title=Doorbell works fine when using -f segment

ffmpeg is so powerful but man does it take a ton of trial and error to figure out.

noelhibbard commented 3 years ago

I finally gave up and just wrote an app that watches for the end events and then waits for the file in the clips folder to finish writing and then I use ffmpeg to rotate the image. It's instant and takes no CPU because it's only changing the metadata. Problem solved.

blakeblackshear commented 3 years ago

@noelhibbard can you post the command you were trying with the -f segment args? One feature I have been considering for the future is "post processors" to modify mp4 files afterwards, and this would fit in that use case.

truxntrax commented 3 years ago

@noelhibbard can you share your solution please?

noelhibbard commented 3 years ago

@blakeblackshear I could never get it to work in conjunction with -f segment so I settled for a post process. The command I am using to rotate 90cw is:

ffmpeg -i in.mp4 -metadata:s:v rotate="-90" -c:v copy out.mp4

You would think -90 would be 90ccw. Seems backwards. Hahaha

My post process isn't working well though. It's hard to know for sure if it's done writing and it's looking like events get consolidated? An integrated post process would be awesome. 

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

maxi1134 commented 1 year ago

This is still not something easy to achieve in the latest 0.12.0beta1 version.

How can one restream a rotated image?

NickM-27 commented 1 year ago

Not sure what you mean, the ffmpeg args are there to rotate it. Go2rtc docs also cover how to setup a stream in its config which has the rotate parameter

maxi1134 commented 1 year ago

the ffmpeg args are there to rotate it

Could you link to a doc page? I tried looking up "transpose" and "rotate" with no luck on the documentation site.

I am unsure where to add the arguments.

NickM-27 commented 1 year ago

You would add them to ffmpeg -> output_args -> detect and ffmpeg -> output_args -> record along with the existing args that are there.

https://www.baeldung.com/linux/ffmpeg-rotate-video

Or you can use go2rtc with #rotate=x at the end of the stream and use that as the source for frigates stream.

maxi1134 commented 1 year ago

You would add them to ffmpeg -> output_args -> detect and ffmpeg -> output_args -> record along with the existing args that are there.

Thanks for those two examples! Unfortunately, my question pertains to "restream"/"rtsp" feeds.

My end-goal is to have the image in the right orientation when viewing it on home-assistant

NickM-27 commented 1 year ago

Yes, then setup the stream with frigate and the restream role and add #rotate=x (x being the degrees) to the end of the stream.

maxi1134 commented 1 year ago

I'm deeply sorry, but I am not sure to understand what you mean by "end of the stream".

Do you mean in the URL used to access the stream from Hass?

NickM-27 commented 1 year ago

No, I mean at the end of the stream used in the frigate config

camera_name:
  ffmpeg:
    inputs:
      - path: "rtsp://ip:554/url#rotate=x"
        roles:
          - restream
      - path: "rtsp://localhost:8554/camera_name"
        roles:
          - detect
          - record
maxi1134 commented 1 year ago

No, I mean at the end of the stream used in the frigate config

First of all, Thanks a lot for the assistance!

This is what I ended up with;

  faceshot_camera_rotated:
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
        - -use_wallclock_as_timestamps
        - "1"
        - -c:v
        - h264_cuvid
        #- hevc_cuvid
      inputs: 
         - path: rtsp://localhost:8554/faceshot_camera_rotated
           roles: 
            - record
         - path: "rtsp://admin:password@192.168.50.9:8554/live0#rotate90" 
           roles:
            - restream
         - path:  rtsp://admin:password@192.168.50.9:8554/live1 
           roles:
            - detect
    detect:
      width: 432
      height: 768
      fps: 5
    snapshots:
      # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
      # This value can be set via MQTT and will be updated in startup based on retained value
      enabled: True
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: False
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: True
      # Optional: crop the snapshot (default: shown below)
      crop: False
      required_zones: []
      # Optional: height to resize the snapshot to (default: original size)
      # Optional: Restrict snapshots to objects that entered any of the listed zones (default: no required zones)
      # Optional: Camera override for retention settings (default: global values)
      retain:
        # Required: Default retention days (default: shown below)
        default: 10
        # Optional: Per object retention days
        objects:
          person: 60
    record:
        # Optional: Enable recording (default: shown below)
        enabled: true
        # Optional: Number of days to retain (default: shown below)
        #retain_days: 60
        # Optional: Event recording settings
        events:
          # Optional: Enable event recording retention settings (default: shown below)
          # Optional: Maximum length of time to retain video during long events. (default: shown below)
          # NOTE: If an object is being tracked for longer than this amount of time, the cache
          #       will begin to expire and the resulting clip will be the last x seconds of the event unless retain_days under record is > 0.
          # max_seconds: 60000
          # Optional: Number of seconds before the event to include in the event (default: shown below)
          pre_capture: 10
          # Optional: Number of seconds after the event to include in the event (default: shown below)
          post_capture: 10
          # Optional: Objects to save event for. (default: all tracked objects)
          objects:
            - person
          # Optional: Restrict event to objects that entered any of the listed zones (default: no required zones)
          required_zones: []
          # Optional: Retention settings for event
          retain:
            # Required: Default retention days (default: shown below)
            default: 10
            # Optional: Per object retention days
            objects:
              person: 60
    #objects:
    objects:
      filters:
        person:
          min_area: 20000 #previouslyt 480000
         # mask:
            # bonne coordene  - 1296,2304,1296,1584,1296,1425,646,1413,0,1455,0,2304
            # - 1296,2304,1296,1659,1296,1807,1296,1397,699,1426,615,1381,550,1433,0,1415,0,2304
    zones:
      facecam_zone:
        coordinates: 1296,2304,1296,0,0,0,0,2304

Unfortunately, the image is still not rotated in frigate or home-assistant.

NickM-27 commented 1 year ago

You didn't follow the example, you have rotate90 but it needs to be rotate=90

maxi1134 commented 1 year ago

You didn't follow the example, you have rotate90 but it needs to be rotate=90

Even with that correction, the logs show;


2023-01-02 21:12:56.883929601  02:12:56.883 ERR github.com/AlexxIT/go2rtc/cmd/streams/producer.go:51 > error="exec: exit status 1"
2023-01-02 21:12:56.883933171  02:12:56.883 WRN [rtsp] error="sources unavailable: 1" stream=faceshot_camera_rotated

I also tried

- path: "rtsp://admin:password@192.168.50.9:8554/live0#transpose=1"

With no log error, but no rotation

NickM-27 commented 1 year ago

In that case I'd recommend creating a manual go2rtc config as described in the docs and adding it there

You can see an example of it here https://github.com/AlexxIT/go2rtc#source-ffmpeg

maxi1134 commented 1 year ago

I am already running Go2RTC and Frigate 0.11

The goal here is to eliminate the need for two docker containers.

But thanks anyway. I'll continue with my current setup until Frigate is more mature and can do a simple rotation on a FFMPEG transcode.

NickM-27 commented 1 year ago

Go2rtc is already built into frigate 0.12, you can provide a manual config file for the embedded go2rtc per our beta docs. I have a working example of this rotation on my own setup and it's working just fine.

maxi1134 commented 1 year ago

I have a working example

Could you provide the working sample?

That would be much appreciated

NickM-27 commented 1 year ago

I have a working example

Could you provide the working sample?

That would be much appreciated

I found two ways to do it, you can either follow https://deploy-preview-4055--frigate-docs.netlify.app/configuration/live#webrtc-extra-configuration to edit the embedded go2rtc config and add the stream with #video=h264#rotate=x

or

you can setup frigate like:

  faceshot_camera_rotated:
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer
        - -flags
        - low_delay
        - -strict
        - experimental
        - -fflags
        - +genpts+discardcorrupt
        - -use_wallclock_as_timestamps
        - "1"
        - -c:v
        - h264_cuvid
        #- hevc_cuvid
      inputs: 
         - path: rtsp://localhost:8554/faceshot_camera_rotated
           roles: 
            - record
         - path: "ffmpeg:rtsp://admin:password@192.168.50.9:8554/live0#video=h264#rotate=90" 
           roles:
            - restream
         - path:  rtsp://admin:password@192.168.50.9:8554/live1 
           roles:
            - detect
    restream:
      force_audio: false
NickM-27 commented 1 year ago

the second method didn't work with some of my cameras due to their URL and the password escaping that is done, I'd suggest just doing it the first way

maxi1134 commented 1 year ago

Thanks for the help @NickM-27 , could you provide a sample for the first method?

I read that page but must admit I still don't see how to set a camera there

NickM-27 commented 1 year ago

You said you have go2rtc setup outside frigate right?

Just take that config file and follow the instructions on those docs to set the config for the built in go2rtc. Then access the camera using localhost:8554/name_set_in_config

maxi1134 commented 1 year ago

I did some more testing.

And I get this in the logs

2023-01-15 21:58:59.250949307    Metadata:
2023-01-15 21:58:59.250950367      title           : Session Streamed by LIBZRTSP
2023-01-15 21:58:59.250951167      comment         : live0
2023-01-15 21:58:59.250951907    Duration: N/A, start: 1.200000, bitrate: N/A
2023-01-15 21:58:59.250952767    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1280x720, 10 tbr, 90k tbn
2023-01-15 21:58:59.250953637  Filtergraph 'transpose=1' was defined for video output stream 0:2 but codec copy was selected.
2023-01-15 21:58:59.250954357  Filtering and streamcopy cannot be used together.
2023-01-15 21:58:59.251278273  02:58:59.251 ERR github.com/AlexxIT/go2rtc/cmd/streams/producer.go:51 > error="exec: exit status 1" url=ffmpeg:rtsp://*:*@192.168.50.9:8554/live0#video=h264#rotate=90#video=h264#video=copy#audio=aac#audio=opus
2023-01-15 21:58:59.251281583  02:58:59.251 DBG [exec] run url="exec:ffmpeg -hide_banner -allowed_media_types video+audio -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i rtsp://*:*@192.168.50.9:8554/live0 -map 0:v:0? -c:v:0 libx264 -g 50 -profile:v:0 high -level:v:0 4.1 -preset:v:0 superfast -tune:v:0 zerolatency -map 0:v:0? -c:v:1 libx264 -g 50 -profile:v:1 high -level:v:1 4.1 -preset:v:1 superfast -tune:v:1 zerolatency -map 0:v:0? -c:v:2 copy -map 0:a:0? -c:a:0 aac -map 0:a:0? -c:a:1 libopus -ar:a:1 48000 -ac:a:1 2 -vf transpose=1 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://localhost:8554/28c71e66165a1ed3357ec7d3f8221352"

With the following path:

  faceshot_camera:
    ffmpeg:
      hwaccel_args: preset-nvidia-h264

      # input_args:
        # - -avoid_negative_ts
        # - make_zero
        # - -fflags
        # - nobuffer
        # - -flags
        # - low_delay
        # - -strict
        # - experimental
        # - -fflags
        # - +genpts+discardcorrupt
        # - -use_wallclock_as_timestamps
        # - "1"
        # - -c:v
        # - h264_cuvid
        # #- hevc_cuvid
      inputs: 
         - path: "rtsp://*:*@192.168.50.9:8554/live0#video=h264#rotate=90#video=h264"
           roles: 
            - record
            - restream
         - path: "rtsp://*:*@192.168.50.9:8554/live1#video=h264#rotate=90#video=h264"
           roles:
            - detect
    detect:
      width: 432
      height: 768
      fps: 5
    snapshots:
      # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
      # This value can be set via MQTT and will be updated in startup based on retained value
      enabled: True
      # Optional: print a timestamp on the snapshots (default: shown below)
      timestamp: False
      # Optional: draw bounding box on the snapshots (default: shown below)
      bounding_box: True
      # Optional: crop the snapshot (default: shown below)
      crop: False
      required_zones: []
      # Optional: height to resize the snapshot to (default: original size)
      # Optional: Restrict snapshots to objects that entered any of the listed zones (default: no required zones)
      # Optional: Camera override for retention settings (default: global values)
      retain:
        # Required: Default retention days (default: shown below)
        default: 10
        # Optional: Per object retention days
        objects:
          person: 60
    record:
        # Optional: Enable recording (default: shown below)
        enabled: true
        # Optional: Number of days to retain (default: shown below)
        #retain_days: 60
        # Optional: Event recording settings
        events:
          # Optional: Enable event recording retention settings (default: shown below)
          # Optional: Maximum length of time to retain video during long events. (default: shown below)
          # NOTE: If an object is being tracked for longer than this amount of time, the cache
          #       will begin to expire and the resulting clip will be the last x seconds of the event unless retain_days under record is > 0.
          # max_seconds: 60000
          # Optional: Number of seconds before the event to include in the event (default: shown below)
          pre_capture: 10
          # Optional: Number of seconds after the event to include in the event (default: shown below)
          post_capture: 10
          # Optional: Objects to save event for. (default: all tracked objects)
          objects:
            - person
          # Optional: Restrict event to objects that entered any of the listed zones (default: no required zones)
          required_zones: []
          # Optional: Retention settings for event
          retain:
            # Required: Default retention days (default: shown below)
            default: 10
            # Optional: Per object retention days
            objects:
              person: 60
    #objects:
    objects:
      filters:
        person:
          min_area: 20000 #previouslyt 480000
         # mask:
            # bonne coordene  - 1296,2304,1296,1584,1296,1425,646,1413,0,1455,0,2304
            # - 1296,2304,1296,1659,1296,1807,1296,1397,699,1426,615,1381,550,1433,0,1415,0,2304
    zones:
      facecam_zone:
        coordinates: 1296,2304,1296,0,0,0,0,2304

It seems like this error: "Filtering and streamcopy cannot be used together"

Refers to this in the logs.


2023-01-15 21:58:59.251278273  02:58:59.251 ERR github.com/AlexxIT/go2rtc/cmd/streams/producer.go:51 > error="exec: exit status 1" url=ffmpeg:rtsp://*:*@192.168.50.9:8554/live0#video=h264#rotate=90#video=h264#video=copy#audio=aac#audio=opus

Can we, in any way, not append "#video=copy"?

scaytrase commented 1 year ago

Can we create a man\faq page for this? I've faced the same exact problem for my rotated reolink camera, still struggling. I can open separate issue for that if it will be considered

NickM-27 commented 1 year ago

Can we create a man\faq page for this? I've faced the same exact problem for my rotated reolink camera, still struggling. I can open separate issue for that if it will be considered

the suggestion in 0.12 is to use go2rtc to create a rotated feed instead of doing it manually in frigate

scaytrase commented 1 year ago

Yea, I kinda understand but is there more details on the topic or just we should learn go2rtc by ourselves?

NickM-27 commented 1 year ago

Yea, I kinda understand but is there more details on the topic or just we should learn go2rtc by ourselves?

there's a whole page of documentation for go2rtc on the go2rtc repo https://github.com/AlexxIT/go2rtc#source-ffmpeg