Unable to connect to Mobotix Streams using FFMpeg on Jetson Nano

[Kehvarl]

I'm trying to set up Viseron on a Jetson Nano to act as an NVR for some Mobotix M26 cameras. The streams from these cameras can be accessed either using their onvif profile or through their faststream url, which is the preferred method.

I've tested that the faststream URL opens in VLC and also can be recorded using FFMpeg from the command line, which works without issue. However, I'm having problems getting Viseron to connect.

I receive this error message:

[2023-06-30 16:20:24] [INFO    ] [viseron.components] - Setting up domain camera for component ffmpeg with identifier camera_1, attempt 9
[2023-06-30 16:20:24] [ERROR   ] [viseron.components] - Domain camera for component ffmpeg is not ready. Retrying in 90 seconds. Error: FFprobe could not connect to stream. Output: {'error': {'code': -1094995529, 'string': 'Invalid data found when processing input'}}
[2023-06-30 16:21:23] [WARNING ] [viseron.components] - Domain nvr for component nvr with identifier camera_1 is still waiting for dependencies: ['domain: camera, identifier: camera_1', 'domain: object_detector, identifier: camera_1', 'domain: motion_detector, identifier: camera_1']
[2023-06-30 16:21:23] [WARNING ] [viseron.components] - Domain object_detector for component darknet with identifier camera_1 is still waiting for dependencies: ['domain: camera, identifier: camera_1', 'domain: motion_detector, identifier: camera_1']
[2023-06-30 16:21:23] [WARNING ] [viseron.components] - Domain motion_detector for component mog2 with identifier camera_1 is still waiting for dependencies: ['domain: camera, identifier: camera_1']

If I run FFProbe from the command line as such: ffprobe -i "http://<user>:<pass>@<IP>:80/control/faststream.jpg?stream=full" I get a valid response:

ffprobe version 3.4.11-0ubuntu0.1 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
[NULL @ 0x55a60bff10] Opening 'http://**********:**********@**********:80/control/faststream.jpg?stream=full' for reading
[http @ 0x55a609caf0] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[http @ 0x55a609caf0] request: GET /control/faststream.jpg?stream=full HTTP/1.1
User-Agent: Lavf/57.83.100
Accept: */*
Range: bytes=0-
Connection: close
Host: 10.0.5.42:80
Icy-MetaData: 1

[http @ 0x55a609caf0] header='HTTP/1.1 401 Unauthorized'
[http @ 0x55a609caf0] http_code=401
[http @ 0x55a609caf0] header='Server: mxhttpd/2.19-MX Sep 21 2021'
[http @ 0x55a609caf0] header='Content-type: text/html'
[http @ 0x55a609caf0] header='Date: Fri, 30 Jun 2023 23:22:33 GMT'
[http @ 0x55a609caf0] header='Last-modified: Fri, 30 Jun 2023 23:22:33 GMT'
[http @ 0x55a609caf0] header='Accept-Ranges: bytes'
[http @ 0x55a609caf0] header='Connection: close'
[http @ 0x55a609caf0] header='Content-length: 7557'
[http @ 0x55a609caf0] header='X-Frame-Options: SAMEORIGIN'
[http @ 0x55a609caf0] header='WWW-Authenticate: Basic realm="MOBOTIX Camera User"'
[http @ 0x55a609caf0] header='WWW-Authenticate: Digest realm="MOBOTIX Camera User", algorithm=MD5, qop="auth", stale=false, nonce="000031cigmifhfnbohfgllhcjfbeinahccndil"'
[http @ 0x55a609caf0] header=''
[http @ 0x55a609caf0] request: GET /control/faststream.jpg?stream=full HTTP/1.1
User-Agent: Lavf/57.83.100
Accept: */*
Range: bytes=0-
Connection: close
Host: 10.0.5.42:80
Icy-MetaData: 1
Authorization: Digest username="*****", realm="MOBOTIX Camera User", nonce="000031cigmifhfnbohfgllhcjfbeinahccndil", uri="/control/faststream.jpg?stream=full", response="927f039966f47c92afe9ee0228924c72", algorithm="MD5", qop="auth", cnonce="cce0e9128c19d3e1", nc=00000001

[http @ 0x55a609caf0] header='HTTP/1.0 200 OK'
[http @ 0x55a609caf0] http_code=200
[http @ 0x55a609caf0] header='Content-type: multipart/x-mixed-replace; boundary="MOBOTIX_Fast_Serverpush"'
[http @ 0x55a609caf0] header=''
Probing mpjpeg score:100 size:2048
[mpjpeg @ 0x55a60bff10] Format mpjpeg probed with size=2048 and score=100
[mpjpeg @ 0x55a60bff10] Before avformat_find_stream_info() pos: 0 bytes read:4000 seeks:0 nb_streams:1
[mjpeg @ 0x55a60c2a40] marker=d8 avail_size_in_buf=131606
[mjpeg @ 0x55a60c2a40] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x55a60c2a40] marker=e0 avail_size_in_buf=131604
[mjpeg @ 0x55a60c2a40] marker parser used 16 bytes (128 bits)
[mjpeg @ 0x55a60c2a40] marker=fe avail_size_in_buf=131586
[mjpeg @ 0x55a60c2a40] marker parser used 1638 bytes (13104 bits)
[mjpeg @ 0x55a60c2a40] marker=fe avail_size_in_buf=129946
[mjpeg @ 0x55a60c2a40] marker parser used 175 bytes (1400 bits)
[mjpeg @ 0x55a60c2a40] marker=db avail_size_in_buf=129769
[mjpeg @ 0x55a60c2a40] index=0
[mjpeg @ 0x55a60c2a40] qscale[0]: 2
[mjpeg @ 0x55a60c2a40] marker parser used 67 bytes (536 bits)
[mjpeg @ 0x55a60c2a40] marker=db avail_size_in_buf=129700
[mjpeg @ 0x55a60c2a40] index=1
[mjpeg @ 0x55a60c2a40] qscale[1]: 5
[mjpeg @ 0x55a60c2a40] marker parser used 67 bytes (536 bits)
[mjpeg @ 0x55a60c2a40] marker=c0 avail_size_in_buf=129631
[mjpeg @ 0x55a60c2a40] Changing bps from 0 to 8
[mjpeg @ 0x55a60c2a40] sof0: picture: 1280x960
[mjpeg @ 0x55a60c2a40] component 0 2:2 id: 0 quant:0
[mjpeg @ 0x55a60c2a40] component 1 1:1 id: 1 quant:1
[mjpeg @ 0x55a60c2a40] component 2 1:1 id: 2 quant:1
[mjpeg @ 0x55a60c2a40] pix fmt id 22111100
[mjpeg @ 0x55a60c2a40] marker parser used 17 bytes (136 bits)
[mjpeg @ 0x55a60c2a40] marker=c4 avail_size_in_buf=129612
[mjpeg @ 0x55a60c2a40] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x55a60c2a40] marker=c4 avail_size_in_buf=129579
[mjpeg @ 0x55a60c2a40] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x55a60c2a40] marker=c4 avail_size_in_buf=129396
[mjpeg @ 0x55a60c2a40] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x55a60c2a40] marker=c4 avail_size_in_buf=129363
[mjpeg @ 0x55a60c2a40] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x55a60c2a40] escaping removed 5993 bytes
[mjpeg @ 0x55a60c2a40] marker=da avail_size_in_buf=129180
[mjpeg @ 0x55a60c2a40] marker parser used 123187 bytes (985496 bits)
[mjpeg @ 0x55a60c2a40] marker=d9 avail_size_in_buf=0
[mjpeg @ 0x55a60c2a40] decode frame unused 0 bytes
[mpjpeg @ 0x55a60bff10] All info found
[mpjpeg @ 0x55a60bff10] stream 0: start_time: -368934881474191040.000 duration: -368934881474191040.000
[mpjpeg @ 0x55a60bff10] format: start_time: -9223372036854.775 duration: -9223372036854.775 bitrate=0 kb/s
[mpjpeg @ 0x55a60bff10] After avformat_find_stream_info() pos: 131687 bytes read:131687 seeks:0 frames:1
Input #0, mpjpeg, from 'http://**********:**********@********:80/control/faststream.jpg?stream=full':
  Duration: N/A, bitrate: N/A
    Stream #0:0, 1, 1/25: Video: mjpeg, 1 reference frame, yuvj420p(pc, bt470bg/unknown/unknown, center), 1280x960 [SAR 1:1 DAR 4:3], 0/1, 25 tbr, 25 tbn, 25 tbc
[AVIOContext @ 0x55a6098780] Statistics: 131687 bytes read, 0 seeks

Likewise, using FFMpeg from the commandlline: ffmpeg -f mxg -vsync 0 -i "http://**********:**********@**********/control/faststream.jpg?stream=full" Desktop/test.mp4 Works to create a video clip, so I know the FFMpeg on the Jetson can handle the cameras.

My current Camera Domain configuration.

## Start by adding some cameras
ffmpeg:
  camera:
    camera_1:  # This value has to be unique across all cameras
      ffmpeg_loglevel: debug
      ffprobe_loglevel: debug
      name: Cam 1
      host: <ip>
      port: 80
      path: /control/faststream.jpg?stream=full
      input_args: 
        - f
        - mxg
        - vsync 
        - 0
      fps: 25
      username: <user>
      password: <password>

[roflcoopter]

It appears to be an MJPEG stream so you need to set some additional config options for the camera. Try this config and see if it works any better:

ffmpeg:
  camera:
    camera_1:  # This value has to be unique across all cameras
      ffmpeg_loglevel: debug
      ffprobe_loglevel: debug
      name: Cam 1
      host: <ip>
      port: 80
      path: /control/faststream.jpg?stream=full
      input_args: 
        - f
        - mxg
        - vsync 
        - 0
      fps: 25
      username: <user>
      password: <password>
      stream_format: mjpeg
      protocol: http

If that does not work, please provide the full log during Viserons startup

[Kehvarl]

Still can't get it to load the view. I did have to make a couple more changes to the config to get to this point (Viseron Startup Output attached)

New error:

[2023-07-03 11:48:47] [ERROR   ] [viseron.components.ffmpeg.camera.camera_1] - FFmpeg process has exited
[2023-07-03 11:48:52] [ERROR   ] [viseron.components.ffmpeg.camera.camera_1] - Restarting frame pipe

Current Config

ffmpeg:
  camera:
    camera_1:  # This value has to be unique across all cameras
      ffmpeg_loglevel: trace
      ffprobe_loglevel: debug
      name: Cam1
      host: <IP>
      port: 80
      path: /control/faststream.jpg?stream=full&fps=10
      fps: 10
      input_args: 
[Viseron_Startup.txt](https://github.com/roflcoopter/viseron/files/11940279/Viseron_Startup.txt)

        - f
        - mxg
        - vsync 
        - "0"
      username: <user>
      password: <password>
      stream_format: mjpeg
      protocol: http

[roflcoopter]

Could you enable debug logging? This will let me see the generated FFmpeg command

logger:
  default_level: debug

[Kehvarl]

After enabling logging, I went through again and tried to fix the problem. My new config is:

ffmpeg:
  camera:
    camera_1:  # This value has to be unique across all cameras
      ffmpeg_loglevel: trace
      ffprobe_loglevel: debug
      name: Cam1
      host: !secret camera1_ip
      port: 80
      path: /control/faststream.jpg?stream=full&fps=10
      fps: 10
      input_args: 
        - -vsync 
        - "0"
        - -f
        - mxg
      username: !secret camera1_username
      password: !secret camera1_password
      #stream_format: mjpeg
      rtsp_transport: http
      protocol: http

With these changes, according to the logging, the FFMpeg command is: [2023-07-05 09:28:35] [DEBUG ] [viseron.components.ffmpeg.stream.camera_1] - FFmpeg decoder command: ffmpeg_camera_1 -hide_banner -loglevel trace -vsync 0 -f mxg -rtsp_transport http -i http://*****:*****@<IP>:80/control/faststream.jpg?stream=full&fps=10 -f segment -segment_time 5 -reset_timestamps 1 -strftime 1 -c:v copy /segments/camera_1/%Y%m%d%H%M%S.mp4 -vf fps=1.0 -f rawvideo -pix_fmt nv12 pipe:1

Dropping into the shell, I tried this command directly: ffmpeg -hide_banner -loglevel trace -vsync 0 -f mxg -rtsp_transport http -i http://*****:*****@<IP>:80/control/faststream.jpg?stream=full&fps=10 -f segment -segment_time 5 -reset_timestamps 1 -strftime 1 -c:v copy /segments/camera_1/%Y%m%d%H%M%S.mp4 -vf fps=1.0 -f rawvideo -pix_fmt nv12

It fails with an error:

Option rtsp_transport not found.

After removing -rtsp_transport http: ffmpeg -hide_banner -loglevel trace -vsync 0 -f mxg -i "http://*****:*****@<IP>:80/control/faststream.jpg?stream=full&fps=10" -f segment -segment_time 5 -reset_timestamps 1 -strftime 1 -c:v copy /segments/camera_1/%Y%m%d%H%M%S.mp4 -vf fps=1.0 -f rawvideo -pix_fmt nv12

It fails with repeated error:

cur_dts is invalid (this is harmless if it occurs once at the start per stream)
    Last message repeated 109 times

Once I ctrl+c the stream, I then get:

Finishing stream 0:1 without any data written to it.
detected 4 logical cores
[graph_0_in_0_1 @ 0x55abbafa80] Setting 'time_base' to value '1/8000'
[graph_0_in_0_1 @ 0x55abbafa80] Setting 'sample_rate' to value '8000'
[graph_0_in_0_1 @ 0x55abbafa80] Setting 'sample_fmt' to value 's16'
[graph_0_in_0_1 @ 0x55abbafa80] Setting 'channel_layout' to value '0x4'
[graph_0_in_0_1 @ 0x55abbafa80] tb:1/8000 samplefmt:s16 samplerate:8000 chlayout:0x4
[format_out_0_1 @ 0x55abbafe40] Setting 'sample_fmts' to value 'fltp'
[format_out_0_1 @ 0x55abbafe40] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
[format_out_0_1 @ 0x55abbafe40] auto-inserting filter 'auto_resampler_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_1'
[AVFilterGraph @ 0x55abbcbda0] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed
[auto_resampler_0 @ 0x55abbb07f0] [SWR @ 0x55abbb1fa0] Using s16p internally between filters
[auto_resampler_0 @ 0x55abbb07f0] ch:1 chl:mono fmt:s16 r:8000Hz -> ch:1 chl:mono fmt:fltp r:8000Hz
[aac @ 0x55abbae870] Too many bits 8832.000000 > 6144 per frame requested, clamping to max
[segment @ 0x55abbac170] Selected stream id:0 type:video
[segment @ 0x55abbac170] Opening 'Desktop/20230705095008.mp4' for writing
[file @ 0x55abb799a0] Setting default whitelist 'file,crypto'
[mp4 @ 0x55abbcadb0] Could not find tag for codec mxpeg in stream #0, codec not currently supported in container
[AVIOContext @ 0x55abb79890] Statistics: 0 seeks, 0 writeouts
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:1 --

Apparently Mxpeg format segments can't be stored in an MP4 container with the -c:v copy argument since mp4 can't hold mxpeg streams. If I force the filename to Mxpeg, it fails as well, with an argument about no streams in the file.

After experimenting a bit I found that removing -c:v copy ffmpeg -hide_banner -loglevel trace -vsync 0 -f mxg -i "http://*****:*****@<IP>:80/control/faststream.jpg?stream=full&fps=10" -f segment -segment_time 5 -reset_timestamps 1 -strftime 1 Desktop/%Y%m%d%H%M%S.mp4 -f rawvideo -pix_fmt nv12 Works, and creates MP4 video files.

Attached is the output with logging enabled.
Viseron.txt

[roflcoopter]

I see, thanks for the great report! It appears that MXPEG (actually never heard of this before) needs some special treatment.

Will get right on it

[roflcoopter]

The dev tag now contains a new config option, raw_command that you can use to run your custom commands.

I figured this was the best solution since i do not have a camera to test out my changes with. Let me know if you need any help to get the configuration correct. I added a section to the docs which hopefully explains it well enough: https://dev--viseron.netlify.app/components-explorer/components/ffmpeg#raw-command

[Kehvarl]

Is this change up on the jetson-nano-viseron:dev tag as well as the regular viseron:dev tag? I'm getting an error that raw_command is not a valid tag:

[2023-07-11 08:48:39] [ERROR   ] [viseron.components] - Error validating config for domain camera and component ffmpeg: extra keys not allowed @ data['camera_1']['raw_command']. Got None

ffmpeg:
  camera:
    camera_1:  # This value has to be unique across all cameras
      name: Cam1
      host: !secret camera1_ip
      port: 80
      path: /control/faststream.jpg?stream=full&fps=10
      width: 1920
      height: 1080
      fps: 10
      username: !secret camera1_username
      password: !secret camera1_password
      raw_command: |
        ffmpeg -i http://<user>:<pass>@<ip>:80/control/faststream.jpg?stream=full&fps=10 -f segment -segment_time 5 -reset_timestamps 1 -strftime 1 /segments/viseron_vscode_camera/%Y%m%d%H%M%S.mp4 -f rawvideo -pix_fmt nv12

[Kehvarl]

Disregard. I did a new docker image pull and now the command is recognized.

Thank you!