[Camera Support]: Unifi G3 Instant / JPEG snapshot / What settings for detect/recording?

[ozett]

Describe the problem you are having

i am happy to overcome the greenscreen and displaying the snapshot of my standalone running unify G3 instant camera. but i see errors and ffmepg crashes. naturally i must be difficult to get recording or detection video from static snapshots. but as i see snapshots, AI detection must be possible.

The questions: 1) How to increase FPS , means faster grabbing of the snapshot-image. 2) What settings in this specific case for detection and output from the cam?

Version

DEBUG 0.10.1-83481AF

Frigate config file

g3instant:
    detect:
      width: 1920 #640 #704    # 2560
      height: 1080 #480 #576   # 1440
      fps: 5
    ffmpeg:
      inputs:
        - path: http://192.168.14.35/snap.jpeg
          roles:
            - detect

      input_args:
        - -r
        - 5 # << enter FPS here
        - -stream_loop
        - -1
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - +genpts
        - -use_wallclock_as_timestamps
        - '1'
      output_args:
        detect:
          -f rawvideo
          -pix_fmt yuv420p
          -an

        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v libx264 -an

Relevant log output

frigate    | [2022-04-25 07:49:33] watchdog.g3instant             ERROR   : Ffmpeg process crashed unexpectedly for g3instant.
frigate    | [2022-04-25 07:49:33] watchdog.g3instant             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate    | [2022-04-25 07:49:33] frigate.video                  ERROR   : g3instant: Unable to read frames from ffmpeg process.
frigate    | [2022-04-25 07:49:33] frigate.video                  ERROR   : g3instant: ffmpeg process is not running. exiting capture thread...
frigate    | [2022-04-25 07:49:43] watchdog.g3instant             ERROR   : Ffmpeg process crashed unexpectedly for g3instant.
frigate    | [2022-04-25 07:49:43] watchdog.g3instant             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate    | [2022-04-25 07:49:43] frigate.video                  ERROR   : g3instant: Unable to read frames from ffmpeg process.
frigate    | [2022-04-25 07:49:43] frigate.video                  ERROR   : g3instant: ffmpeg process is not running. exiting capture thread...

FFprobe output from your camera

root@ub2004-fri2:/usr/src# ffprobe http://192.168.14.35/snap.jpeg
ffprobe version 4.2.4-1ubuntu0.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, image2, from 'http://192.168.14.35/snap.jpeg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 16704 kb/s
    Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 1920x1080 [SAR 118:118 DAR 16:9], 25 tbr, 25 tbn, 25 tbc

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

Coral version

M.2

Network connection

Wired

Camera make and model

unifi g3 instant

Any other information that may be helpful

tried first in motioneyeOS, and jpeg grabbing is smooth, https://github.com/blakeblackshear/frigate/issues/2530 but want to stick with jpeg grabbing (no rtsp on the g3 instant in standalone-mode) https://github.com/blakeblackshear/frigate/issues/50

[NickM-27]

It's not clear what the issue is here, is the camera working and you want to make it better or are you seeing errors when trying to run the camera in frigate?

[ozett]

@NickM-27 thanks for asking: i have errors for the cam in frigate while running the ffmpeg thread.

The cam only delivers a static snapshot.jpeg in standalone-mode. (overall: cheap cam, good image quality) my config seems to be fine for getting this image and showing in the frigate-gui.

BUT

it does not run the continous AI thread/detection without error. i can not figure out the configuration problem.

i disabled recording, but that does not help showing error for the ffmpeg thread. maybe something with the repeatedly framegrabbing for the static image?

any help appreciated.

seems that the ffmpeg log is hidden within the docker on runtime, but the error shuts down the docker, so difficult to get the logs.. any hints for that, too?

(edit: seems difficult to log external, even it the running frigate log tells me that there is value information in it -> (docker) )

thanx, regards, ozett

[ozett]

is there a better way to get more detail logging for ffmpeg-thread(s) ?

this does not help much:

[ozett]

changed ffmpeg loglevel to debug. But poorly it tells me little to overcome the "errors"

[ozett]

my 100 lines of ffmpeg errors:

[ozett]

i found this https://trac.ffmpeg.org/ticket/7525 and disabled the global hwaccell_args

processing changed now, but still ending with an error

[blakeblackshear]

It's just grabbing a single frame and exiting. You will need to find the right ffmpeg command to pull a static image repeatedly into a stream. You will also need to disable rtmp or encode it to h264.

[ozett]

You will need to find the right ffmpeg command to pull a static image repeatedly

ok, thanks for pointing me in the right direction, i will now sail and surf the internet. and see what i can catch and test. i will report on success.. hopefully

(rtmp and encoding already set off) 😉

[ozett]

must be possible.. https://stackoverflow.com/questions/44128029/ffmpeg-livestream-from-static-image-and-audio

[ozett]

(some hours later.... 😴 )

its running error-free on a static image which is saved with wget to the media folder.

detection working on static image with this config:

g3instant:
    detect:
      width: 1920 
      height: 1080 
      fps: 15

    ffmpeg:
      inputs:
        - path: /media/frigate/snap.jpeg
          roles:
            - detect

      input_args:
      - -framerate
      - 15
      - -re
      - -thread_queue_size
      - 512
      - -f
      - image2
      - -loop 
      - 1
      - -an

      output_args:
        detect:
          -filter:v fps=15
          -f rawvideo
          -pix_fmt yuv420p
          -an

but the same snapshot.jpeg with ffmepg http-source does not feed the pipe still thread-error with ffmpeg source as http://snapshot.jpeg. maybe too slow from http ? maybe input fps can be adjustable for creation of pipe-stream?

[ozett]

@ the moment it seems to me, that i cannot solve this with any ffmpeg cmd. my knowledge is limited, i tested now some hours. but ffmpeg seems fine in encoding,

only frigate wants faster frame input. maybe ffmpeg cannot grab+encode faster from http-souces? i dont have a clue. maybe it must be solved in the source code of frigate? where the error comes from?

or it needs to copy the python solution of motioneye for static images?

this looks somekind of correct for now (or not?): 😒

[ozett]

last idea for today: change watchdog-timing for the frigate thread ?

https://github.com/blakeblackshear/frigate/blob/08f573aaa5ec0b9bf3d4b40388942254448b902a/frigate/video.py#L156

there seems to be a calculation with FPS to trigger the timeout. maybe i can adjust FPS parameter to have more time on the static image grabbing?

[blakeblackshear]

Based on everything I have seen, there is no way to have ffmpeg re-pull an image from a URL at an interval and create a video stream. It reads it once and then exits. The messages you are seeing from frigate indicate the ffmpeg process has exited and is no longer running. The logs show that is what is happening too. There is nothing that can be done in frigate's code to address this.

[ozett]

ok, cool. thanks for informing me. it runs somehow stable, because now every 10s the tread is restarted and a new image is pulled.

maybe i figure out how to grab the snapshot with other means (motioneyeos? python-thread?) and feed it as -input into the ffmpeg beginning...

otherwise i consider buying another cam with rtsp, or hacking the G3 Instant for rtsp in standanlone-mode. lets see what time i can spent on either option...

thanks again.

[tootai]

Hi all. I opened case #3192 having same problem with a Vivotek PZ6122 camera.

I don't agree with this comment from blakeblackshear

Based on everything I have seen, there is no way to have ffmpeg re-pull an image from a URL at an interval and create a video stream. It reads it once and then exits.

as I get the stream using ffmpeg with ffmpeg -y -framerate 15 -re -thread_queue_size 512 -loop 1 -f image2pipe -an -i http://USER:PASSWD@192.168.10.150/cgi-bin/video.jpg /tmp/video.avi

Console logs: Input #0, image2pipe, from 'http://root:2tai4video@192.168.10.150/cgi-bin/video.jpg': Duration: N/A, bitrate: N/A Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 352x288 [SAR 72:72 DAR 11:9], 15 fps, 15 tbr, 15 tbn, 15 tbc Stream mapping: Stream #0:0 -> #0:0 (mjpeg (native) -> mpeg4 (native)) Press [q] to stop, [?] for help [swscaler @ 0x55bb5f9c6000] deprecated pixel format used, make sure you did set range correctly Output #0, avi, to '/tmp/video.avi': Metadata: ISFT : Lavf58.45.100 Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 352x288 [SAR 1:1 DAR 11:9], q=2-31, 200 kb/s, 15 fps, 15 tbn, 15 tbc Metadata: encoder : Lavc58.91.100 mpeg4 Side data: cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A frame= 154 fps=4.3 q=11.7 Lsize= 394kB time=00:00:10.26 bitrate= 314.2kbits/s speed=0.288x
video:384kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.421876%

Question is why frigate can't get this video stream ?

[ozett]

hmmm, may i re-acitvate my cam?

as I get the stream using ffmpeg with ffmpeg -y -framerate 15 -re -thread_queue_size 512 -loop 1 -f image2pipe -an -i http://USER:PASSWD@192.168.10.150/cgi-bin/video.jpg /tmp/video.avi

you are writing into an avi-file, right?

but freagate-ai-thread needs constant input out of a pipe, i think. and that pipe must be constantly (and fast) feeded with images. from that ffmpeg cmd-line.

last time i thought, grabbing images with ffmpeg could not feed this pipe fast enough... i thought of feeding the pipe from other sources, but maybe there is still a way to use ffmpeg.. ?

[tootai]

Yes, writing to avi file. Also tested with mpeg. It it needs constant input, perhaps sending the stream to a tcp socket could do the job?

If I replace image2pipe with image2 I get [image2 @ 0x561895edf280] Could not find codec parameters for stream 0 (Video: mjpeg, none(bt470bg/unknown/unknown)): unspecified size Consider increasing the value for the 'analyzeduration' and 'probesize' options Input #0, image2, from 'http://root:2tai4video@192.168.10.150/cgi-bin/video.jpg': Duration: 00:00:00.07, start: 0.000000, bitrate: N/A Stream #0:0: Video: mjpeg, none(bt470bg/unknown/unknown), 15 tbr, 15 tbn, 15 tbc Output #0, avi, to '/tmp/video.avi': Output file #0 does not contain any stream

[ozett]

huhmmm, looks like -image2pipe is for grabbing images from a pipe - into ffmpeg, while -image2 is grabbing images with ffmpeg...

https://ffmpeg.org/faq.html#toc-How-do-I-encode-single-pictures-into-movies_003f

but nonetheless i still think, that the keypoint is getting images fast into the 2nd pipe (ai-thread of frigate). maybe that thread can be feed by ffmpeg with buffer or some other routinge (motioneye-os).

but there a way to much interdependant-options in ffmpeg...

[ozett]

these threads are running, while i have an image from the "snapshot" cam in the gui...

root@ub2004-fri2:~# ps xa | grep ffmpeg
 834829 ?        Ss     0:00 ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1920x1080 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:
 834835 ?        Ss     0:00 ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 15 -bf 0 pipe:
 834935 ?        Ssl    0:00 ffmpeg -hide_banner -loglevel debug -framerate 25 -f image2 -stream_loop -1 -an -i http://192.168.14.35/snap.jpeg -r 15 -s 1920x1080 -f rawvideo -pix_fmt yuv420p -an -preset ultrafast -b:v 600k pipe:
 834955 pts/4    S+     0:00 grep --color=auto ffmpeg

1st & 2nd: birdseye overview

3rd: grabbing from cam, feeding into pipe...

every 10s: error from frigate-thread (maybe from watchdog) https://github.com/blakeblackshear/frigate/blob/08f573aaa5ec0b9bf3d4b40388942254448b902a/frigate/video.py#L218

[blakeblackshear]

https://stackoverflow.com/questions/5026699/generate-a-movie-with-ffmpeg-from-a-changing-still-image-url#comment77307135_5143602

[blakeblackshear]

@tootai it looks like you have an mjpeg stream at that endpoint, not a static jpeg url.

[ozett]

and ffmpeg options changed over the years? or has this all different functions?

flowchart LR 

A("-loop_input") --> B("-loop") --> C("-stream_loop")

docu today:

[blakeblackshear]

The comment on that answer says that option is no longer available and doesn't work because the image is fetched once and cached.

[ozett]

ok, ffmpeg fetches once. thats not enough

any chance to feed the AI-Pipe from outside with jpeg-images in the correct format? what would the format be? rtsp?

maybe using gestreamer to grab the snapshots, convert and pipe into frigate/ffmpeg?

https://forums.developer.nvidia.com/t/streaming-jpeg-to-rtsp/139897

last post has a working pipeline: test-launch "multifilesrc location=tmp/img_%05d.jpg loop=1 caps=image/jpeg,framerate=30/1 ! jpegparse ! identity sync=true ! nvjpegdec ! omxh264enc insert-vui=true insert-sps-pps=1 ! h264parse ! rtph264pay name=pay0 "

seems more an interesting proof-of-concept than of practical use. but seems to make fun...

[tootai]

@blakeblackshear no, it's a static jpeg url. iSpy see it as URL jpeg, as well as I get the jpeg if I open the URL in a browser

[stu1811]

@tootai my G3 Instant outputs a video stream and shows up fine in Frigate. In the device settings in Protect you can enable an rtsp stream.

  heat_cam:
    ffmpeg:
      output_args:
        record: -f segment -segment_time 600 -segment_format yuv420p  -reset_timestamps 1 -strftime 1 -c:v copy
      input_args: -fflags nobuffer -rtsp_transport tcp
      inputs:
        - path: rtsps://192.168.42.1:7441/ABC123?enableSrtp
          roles:
            - detect
            - rtmp
        - path: rtsps://192.168.42.1:7441/ABC12345?enableSrtp
          roles:
            - record
    objects: 
      track:
        - person
        - dog
        - car
        # Optional: save clips configuration
    record:
      enabled: true
      retain:
        days: 15
      events:
        pre_capture: 5
        post_capture: 5
        required_zones: []
        retain:
          # Required: Default retention days (default: shown below)
          default: 30
          # Optional: Per object retention days
          #objects:
          #  person: 15
    snapshots:
      enabled: true
    detect:
      width: 640
      height: 360
      fps: 5

[ozett]

this rtsp(s):// urls are coming from the protect app, not from the cam itself. the shown IP is not from the cam itself?

they changed the firmware (some threads in ubiqutiy-forum and reddit) and rtsp is gone. for sure no rtsp(s) in standalone-mode with my G3-instant cam.

or any hint to activate this again?

[stu1811]

How did you get it into standalone mode? I have an extra I can experiment on.

[ozett]

i fiddelt some time ago, but as far as i remember that was possible while/after adopting. (edit: i think, i configured it in the app/or on the device after login. cant remember exactly, sorry..)

maybe like here discussed https://community.ui.com/questions/G3-Instant-Standalone-Setup-It-works-after-login-to-webgui/1ff56019-9262-45f9-9bd8-5fb4e71e3e96

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.