[Support]: Green camera image after v13 upgrade

[thehhmdb]

Describe the problem you are having

On v12.1 I was using the following to dewarp a fisheye camera, which was working fine: output_args: detect: -vf 'crop=in_w-28:in_h-28:28:28','v360=input=fisheye:output=e:ih_fov=145:iv_fov=145:pitch=3:roll=0:yaw=0','crop=in_w-345-345:in_h-172-142:345:172' -f rawvideo -pix_fmt yuv420p

After the v13.1 upgrade the same camera displays as many green lines.

If I remove the "output_args" completely from the config, the camera will display but without the dewarping (the output is still in the fisheye format)

Also, downgrading back to v12.1 fixes the issue.

Version

v13.1

Frigate config file

mqtt:
  enabled: true
  host: xxx.xxx.xxx.xxx
  port: xxx
cameras:
  cameraname:
    enabled: true
    ffmpeg:
      input_args: -rtsp_transport tcp
      inputs:
        - path: rtsp://xxx.xxx.xxx.xxx:xxx/xxxx
          roles:
            - detect
      output_args:
        detect: -vf 'crop=in_w-28:in_h-28:28:28','v360=input=fisheye:output=e:ih_fov=145:iv_fov=145:pitch=3:roll=0:yaw=0','crop=in_w-345-345:in_h-172-142:345:172' -f rawvideo -pix_fmt yuv420p
objects:
  track:
    - person
  filters:
    person:
      threshold: 0.8
detectors:
  ov:
    type: openvino
    device: AUTO
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml
model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt
record:
  enabled: true
  events:
    retain:
      default: 10
snapshots:
  enabled: true

Relevant log output

-

FFprobe output from your camera

-

Frigate stats

No response

Operating system

Other

Install method

Docker Compose

Coral version

CPU (no coral)

Network connection

Wired

Camera make and model

hikvision

Any other information that may be helpful

Running on TrueNas Scale.

[NickM-27]

there is nothing to go off of here, at the very least would want to see logs and config

[thehhmdb]

there is nothing to go off of here, at the very least would want to see logs and config

Added the config. The logs were not different between versions.

[NickM-27]

nothing changed with ffmpeg itself so it's not clear why this would change, I may need to try it myself and see what happens

[thehhmdb]

I've upgraded to v13.1 again to see if I can reproduce it for a 2nd time, and I get the same result. Here is a screen shot of how the camera output looks if it helps:

[NickM-27]

what happens if you add

detect:
  width: 1280
  height: 720

to your cameras config?

[thehhmdb]

that displays something similar to the green image above, but stretched horizontally.

I also tried 576 which is the steams resolution like: detect: width: 576 height: 576

but it resulted in the same problem as the above screenshot

[NickM-27]

can you try 2 things in this order:

1. open http://frigate_ip:5000/api/config and provide the ffmpeg command
2. try adding hwaccel_args: " " to the camera ffmpeg and see what happens

[thehhmdb]

1. The command is: ffmpeg -hide_banner -loglevel warning -threads 2 -rtsp_transport tcp -i rtsp://xxx.xxx.xxx.xxx:xxx/cameraname -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/cameraname@%Y%m%d%H%M%S%z.mp4 -r 5 -vf fps=5,scale=576:576 -vf crop=in_w-28:in_h-28:28:28,v360=input=fisheye:output=e:ih_fov=145:iv_fov=145:pitch=3:roll=0:yaw=0,crop=in_w-345-345:in_h-172-142:345:172 -f rawvideo -pix_fmt yuv420p pipe:

2. With hwaccel_args: " ", the image still looks green/corrupted like above

[NickM-27]

let's try

detect:
  width: 345
  height: 172

[thehhmdb]

that's generating some errors in the logs:

2024-02-02 23:58:50.531901724 [2024-02-02 23:58:50] frigate.http ERROR : Exception on /cameraname/latest.jpg [GET] 2024-02-02 23:58:50.531909427 Traceback (most recent call last): 2024-02-02 23:58:50.531913571 File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2190, in wsgi_app 2024-02-02 23:58:50.531923369 response = self.full_dispatch_request() 2024-02-02 23:58:50.531927549 File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1486, in full_dispatch_request 2024-02-02 23:58:50.531932660 rv = self.handle_user_exception(e) 2024-02-02 23:58:50.531936550 File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1484, in full_dispatch_request 2024-02-02 23:58:50.531978467 rv = self.dispatch_request() 2024-02-02 23:58:50.531983034 File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1469, in dispatch_request 2024-02-02 23:58:50.532038166 return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) 2024-02-02 23:58:50.532041286 File "/opt/frigate/frigate/http.py", line 1435, in latest_frame 2024-02-02 23:58:50.532043696 frame = current_app.detected_frames_processor.get_current_frame( 2024-02-02 23:58:50.532045815 File "/opt/frigate/frigate/object_processing.py", line 1054, in get_current_frame 2024-02-02 23:58:50.532080933 return self.camera_states[camera].get_current_frame(draw_options) 2024-02-02 23:58:50.532084885 File "/opt/frigate/frigate/object_processing.py", line 484, in get_current_frame 2024-02-02 23:58:50.532088483 frame_copy = cv2.cvtColor(frame_copy, cv2.COLOR_YUV2BGR_I420) 2024-02-02 23:58:50.532093105 cv2.error: OpenCV(4.7.0) /io/opencv/modules/imgproc/src/color.simd_helpers.hpp:108: error: (-215:Assertion failed) sz.width % 2 == 0 && sz.height % 3 == 0 in function 'CvtHelper'

[hawkeye217]

There have been some changes to the way Frigate handles detect resolution and output from ffmpeg in 0.13, so it doesn't surprise me that you are seeing an issue here.

Your original ffmpeg arguments to -vf use the cropping filter, the v360 filter, and another cropping filter (in that order). The dimensions coming out of the final crop are only 345x172, which is not really the ideal size or even a recommended aspect ratio for Frigate.

I would suggest using the lenscorrection filter instead. I have a fisheye lens on my doorbell camera and I use the following in my frigate config file to correct it:

...
      output_args:
        record: preset-record-generic-audio-copy
        detect: -vf lenscorrection=cx=0.5:cy=0.5:k1=-0.290:k2=-0.012
          -f rawvideo -pix_fmt yuv420p
    detect:
      width: 384
      height: 512

384x512 is the native output resolution of the camera's substream, so that's why I have it set as the detect resolution. You'll want to set yours to whatever your camera is outputting.

You'll notice the lenscorrection filter has two main values: k1 and k2. You'll want to adjust these parameters to see what works best for your particular lens. See the ffmpeg docs.

The advantage of using this particular filter is that no cropping is required.

For more on ffmpeg's distortion/lens correction filters, check out this link.

[thehhmdb]

thanks, I'll try that out

[hawkeye217]

An alternative is to still use the v360 fisheye filter, but don't crop or find other ffmpeg filters that can keep the image at the same aspect ratio as your input. YMMV though, that is untested.

[luzik]

It is my issue too. When viewing camera in HA or frigate 'live' - it is ok, but when in http://IP:5000/ (screenshots) or in debug I can see

Here is my config

go2rtc:
  streams:
    wejscie_dzwonek:
      - rtsp://admin:pass@192.168.X.X:554/cam/realmonitor?channel=1&subtype=0
      - ffmpeg:wejscie_dzwonek#audio=aac
cameras:
  wejscie_dzwonek:
    ffmpeg:
      inputs:
        - path: rtsp://admin:pass@192.168.X.X:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
            - record
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
        detect: -vf scale_vaapi=w=1280:h=720,hwdownload,format=yuv420p -f rawvideo
      hwaccel_args:
        - -hwaccel
        - vaapi
        - -hwaccel_device
        - /dev/dri/renderD128
        - -hwaccel_flags
        - allow_profile_mismatch
        - -hwaccel_output_format
        - vaapi

[NickM-27]

@luzik you should just use preset-vaapi it will handle everything for you

[thehhmdb]

I would suggest using the lenscorrection filter instead. I have a fisheye lens on my doorbell camera and I use the following in my frigate config file to correct it:

Using lenscorrection is giving a working image rather than the green lines. I can't get the fisheye dewarping to look as nice as before around the edges, but it's good enough. Thanks for the help

[jurgenweber]

All of my cameras went solid green also, with a bit of play ing around and other reading I turned off all the hardware acceleration config I had for ffmpeg

hwaccel_args: preset-vaapi

just hashed it out.

Any reason to why this would work in v12 and not v13? What are the implications and should I be worried?

[NickM-27]

This issue is not related to what you're seeing and you should make your own issue with logs

[luzik]

@luzik you should just use preset-vaapi it will handle everything for you

Thank You, It worked.