Odroid XU4: Support h264_v4l2m2m hardware accelerated encoder

RatGit commented 5 years ago

Preliminary Docs

I confirm that I have read the CONTRIBUTING guide before opening this issue.

I confirm that I have read the FAQ before opening this issue.

motionEyeOS Version

I am running motionEyeOS version: (insert your version here, e.g. 20180314).

Board Model

I am using the following board/model: (insert your board model here, e.g. Raspberry PI 3B+).

Camera

I am using the following type of camera: (choose from V4L2, MMAL, Network Camera, Fast Network Camera and Simple MJPEG Camera).

My camera model is: (insert your camera model, e.g. Logitech C290).

Network Connection

My motionEyeOS unit is connected to the network via: (choose your network connection, e.g. WiFi, Ethernet).

Peripherals

I am using the following peripherals that I consider relevant to this issue:

(e.g. USB WiFi adapter)
(e.g. External Hard Disk)

Log Files

I consider the following log files relevant to this issue:

(attach e.g. motioneye.log)
(attach e.g. boot.log)

RatGit commented 5 years ago

motionEyeOS Version: 20190427 Board Model: Odroid XU4 Camera: 3 x 720P RTSP Network Cameras Network Connection: Wired Ethernet

Dear MotionEyeOS People,

I've recently tried an Odroid XU4 SBC for this excellent product after previously successfully using R-Pi 3B+ boards. The reason for this was to support three 720P RTSP Cameras that the Pi was struggling with.

Everything has worked well on the XU4 except that H264/OMX support is not available for the FFMPEG binary provided with the XU4 OS image. AFAIK the Odroid XU4 GPU should be able to handle the HW encoding. Would it be possible to provide an FFMPEG binary for the XU4 with H264/OMX enabled?

jasaw commented 5 years ago

I had a quick 10 minutes look into H264 encoding on the XU4. I don't own an Odroid XU4, so I'm not sure what is officially support, and what modifications are needed. I have a few questions though:

From early discussions on various forums, it seem to suggest that it requires some feature to be enabled in the kernel. Is this still the case or is this enabled by default in the mainstream kernel?
It also requires h264_v4l2m2m encoder to be enabled in ffmpeg. It may need Memeka's patches as well. Does anyone know if that's still the case? Does ffmpeg officially support h264_v4l2m2m encoder now? If both kernel and ffmpeg don't require special hacks, it should be pretty easy to support it, provided the encoder is stable and behaves sensibly.

ccrisan commented 5 years ago

@jasaw I can grant you remote login on my testing Odroid XU4 machine, running whatever OS you may be needing. Just let me know and I'll arrange it.

jasaw commented 5 years ago

@ccrisan Thanks for the offer. I'll keep that in mind. I won't be doing any development on motionEyeOS for a couple of months because I'm way too busy with my work.

jasaw commented 5 years ago

@ccrisan I have pulled in the necessary patches from memeka to enable h264 hw encoding on XU4 hardware. In your motion camera-1.conf, you'll need to select h264_v4l2m2m as your video encoder, similar to the omx version. You set bit rate the same way as omx version. Since I don't have an XU4 board, I'm not able to test it. Can you please give it a whirl and let me know how you go? My XU4 H264 branch is here: https://github.com/jasaw/motioneyeos/tree/xu4-hw-h264

ccrisan commented 5 years ago

@jasaw that's great news! Will definitely try it some time during the next weekend.

ccrisan commented 5 years ago

@jasaw I can't seem to make Motion recognize movie_codec mp4:h264_v4l2m2m. Motion log says:

[1:ml1:Camera1] [ERR] [ENC] ffmpeg_set_codec: Could not open codec Invalid argument
[1:ml1:Camera1] [ERR] [ENC] ffmpeg_open: Failed to allocate codec!
[1:ml1:Camera1] [ERR] [EVT] event_ffmpeg_newfile: Error opening context for movie output.

FFmpeg:

# ffmpeg -codecs

ffmpeg version 3.4.5 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 8.2.1 (GNU Toolchain for the A-profile Architecture 8.2-2018.11 (arm-rel-8.26)
  configuration: --enable-cross-compile --cross-prefix=/home/ccrisan/motioneyeos-xu4/output/odroidxu4/host/bin/arm-linux-gnueabihf- --sysroot=/home/ccrisan/motioneyeos-xu4/output/odroidxu4/host/arm-buildroot-linux-gnueabihf/sysroot --host-cc='/home/ccrisan/motioneyeos-xu4/output/odroidxu4/host/bin/ccache /usr/bin/gcc' --arch=arm --target-os=linux --disable-stripping --pkg-config=/home/ccrisan/motioneyeos-xu4/output/odroidxu4/host/bin/pkg-config --disable-static --enable-shared --prefix=/usr --enable-avfilter --disable-version3 --enable-logging --enable-optimizations --disable-extra-warnings --enable-avdevice --enable-avcodec --enable-avformat --enable-network --disable-gray --enable-swscale-alpha --disable-small --enable-dct --enable-fft --enable-mdct --enable-rdft --disable-crystalhd --disable-dxva2 --enable-runtime-cpudetect --disable-hardcoded-tables --disable-mipsdsp --disable-mipsdspr2 --disable-msa --enable-hwaccels --disable-cuda --disable-cuvid --disable-nvenc --disable-avisynth --disable-frei0r --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libdc1394 --disable-libgsm --disable-libilbc --disable-libvo-amrwbenc --disable-symver --disable-doc --enable-gpl --enable-nonfree --enable-ffmpeg --disable-ffplay --disable-ffserver --disable-avresample --disable-ffprobe --disable-postproc --enable-swscale --enable-indevs --enable-alsa --enable-outdevs --enable-pthreads --enable-zlib --disable-bzlib --disable-libfdk-aac --disable-libcdio --disable-gnutls --enable-openssl --disable-libdrm --disable-libopenh264 --enable-libvorbis --enable-muxer=ogg --enable-encoder=libvorbis --disable-vaapi --disable-vdpau --disable-mmal --disable-omx --disable-omx-rpi --disable-libopencv --disable-libopus --disable-libvpx --disable-libass --disable-libbluray --disable-librtmp --disable-libmp3lame --disable-libmodplug --disable-libspeex --enable-libtheora --disable-libwavpack --disable-iconv --disable-libfreetype --disable-fontconfig --disable-libopenjpeg --enable-libx264 --enable-libx265 --disable-x86asm --disable-mmx --disable-sse --disable-sse2 --disable-sse3 --disable-ssse3 --disable-sse4 --disable-sse42 --disable-avx --disable-avx2 --enable-armv6 --enable-vfp --enable-neon --disable-altivec --extra-libs=-latomic --enable-pic --cpu=cortex-a7
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
Codecs:
 D..... = Decoding supported
 .E.... = Encoding supported
 ..V... = Video codec
 ..A... = Audio codec
 ..S... = Subtitle codec
 ...I.. = Intra frame-only codec
 ....L. = Lossy compression
 .....S = Lossless compression
 -------
 D.VI.S 012v                 Uncompressed 4:2:2 10-bit
 D.V.L. 4xm                  4X Movie
 D.VI.S 8bps                 QuickTime 8BPS video
 .EVIL. a64_multi            Multicolor charset for Commodore 64 (encoders: a64multi )
 .EVIL. a64_multi5           Multicolor charset for Commodore 64, extended with 5th color (colram) (encoders: a64multi5 )
 D.V..S aasc                 Autodesk RLE
 D.VIL. aic                  Apple Intermediate Codec
 DEVI.S alias_pix            Alias/Wavefront PIX image
 DEVIL. amv                  AMV Video
 D.V.L. anm                  Deluxe Paint Animation
 D.V.L. ansi                 ASCII/ANSI art
 DEV..S apng                 APNG (Animated Portable Network Graphics) image
 DEVIL. asv1                 ASUS V1
 DEVIL. asv2                 ASUS V2
 D.VIL. aura                 Auravision AURA
 D.VIL. aura2                Auravision Aura 2
 ..V.L. av1                  Alliance for Open Media AV1
 D.V... avrn                 Avid AVI Codec
 DEVI.S avrp                 Avid 1:1 10-bit RGB Packer
 D.V.L. avs                  AVS (Audio Video Standard) video
 DEVI.S avui                 Avid Meridien Uncompressed
 DEVI.S ayuv                 Uncompressed packed MS 4:4:4:4
 D.V.L. bethsoftvid          Bethesda VID video
 D.V.L. bfi                  Brute Force & Ignorance
 D.V.L. binkvideo            Bink video
 D.VI.. bintext              Binary text
 D.VI.S bitpacked            Bitpacked
 DEVI.S bmp                  BMP (Windows and OS/2 bitmap)
 D.V..S bmv_video            Discworld II BMV video
 D.VI.S brender_pix          BRender PIX image
 D.V.L. c93                  Interplay C93
 D.V.L. cavs                 Chinese AVS (Audio Video Standard) (AVS1-P2, JiZhun profile)
 D.V.L. cdgraphics           CD Graphics video
 D.VIL. cdxl                 Commodore CDXL video
 D.V.L. cfhd                 Cineform HD
 DEV.L. cinepak              Cinepak
 D.V.L. clearvideo           Iterated Systems ClearVideo
 DEVIL. cljr                 Cirrus Logic AccuPak
 D.VI.S cllc                 Canopus Lossless Codec
 D.V.L. cmv                  Electronic Arts CMV video (decoders: eacmv )
 D.V... cpia                 CPiA video format
 D.V..S cscd                 CamStudio (decoders: camstudio )
 D.VIL. cyuv                 Creative YUV (CYUV)
 ..V.LS daala                Daala
 D.VILS dds                  DirectDraw Surface image decoder
 D.V.L. dfa                  Chronomaster DFA
 DEV.LS dirac                Dirac (encoders: vc2 )
 DEVIL. dnxhd                VC3/DNxHD
 DEVI.S dpx                  DPX (Digital Picture Exchange) image
 D.V.L. dsicinvideo          Delphine Software International CIN video
 DEVIL. dvvideo              DV (Digital Video)
 D.V..S dxa                  Feeble Files/ScummVM DXA
 D.VI.S dxtory               Dxtory
 D.VIL. dxv                  Resolume DXV
 D.V.L. escape124            Escape 124
 D.V.L. escape130            Escape 130
 D.VILS exr                  OpenEXR image
 DEV..S ffv1                 FFmpeg video codec #1
 DEVI.S ffvhuff              Huffyuv FFmpeg variant
 D.V.L. fic                  Mirillis FIC
 DEVI.S fits                 FITS (Flexible Image Transport System)
 DEV..S flashsv              Flash Screen Video v1
 DEV.L. flashsv2             Flash Screen Video v2
 D.V..S flic                 Autodesk Animator Flic video
 DEV.L. flv1                 FLV / Sorenson Spark / Sorenson H.263 (Flash Video) (decoders: flv ) (encoders: flv )
 D.V..S fmvc                 FM Screen Capture Codec
 D.VI.S fraps                Fraps
 D.VI.S frwu                 Forward Uncompressed
 D.V.L. g2m                  Go2Meeting
 D.V.L. gdv                  Gremlin Digital Video
 DEV..S gif                  GIF (Graphics Interchange Format)
 DEV.L. h261                 H.261
 DEV.L. h263                 H.263 / H.263-1996, H.263+ / H.263-1998 / H.263 version 2 (decoders: h263 h263_v4l2m2m ) (encoders: h263 h263_v4l2m2m )
 D.V.L. h263i                Intel H.263
 DEV.L. h263p                H.263+ / H.263-1998 / H.263 version 2
 DEV.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_v4l2m2m )
 D.VIL. hap                  Vidvox Hap
 DEV.L. hevc                 H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_v4l2m2m ) (encoders: libx265 hevc_v4l2m2m )
 D.V.L. hnm4video            HNM 4 video
 D.VIL. hq_hqa               Canopus HQ/HQA
 D.VIL. hqx                  Canopus HQX
 DEVI.S huffyuv              HuffYUV
 D.V.L. idcin                id Quake II CIN video (decoders: idcinvideo )
 D.VI.. idf                  iCEDraw text
 D.V.L. iff_ilbm             IFF ACBM/ANIM/DEEP/ILBM/PBM/RGB8/RGBN (decoders: iff )
 D.V.L. indeo2               Intel Indeo 2
 D.V.L. indeo3               Intel Indeo 3
 D.V.L. indeo4               Intel Indeo Video Interactive 4
 D.V.L. indeo5               Intel Indeo Video Interactive 5
 D.V.L. interplayvideo       Interplay MVE video
 DEVILS jpeg2000             JPEG 2000
 DEVILS jpegls               JPEG-LS
 D.VIL. jv                   Bitmap Brothers JV video
 D.V.L. kgv1                 Kega Game Video
 D.V.L. kmvc                 Karl Morton's video codec
 D.VI.S lagarith             Lagarith lossless
 .EVI.S ljpeg                Lossless JPEG
 D.VI.S loco                 LOCO
 D.VI.S m101                 Matrox Uncompressed SD
 D.V.L. mad                  Electronic Arts Madcow Video (decoders: eamad )
 D.VI.S magicyuv             MagicYUV video
 D.VIL. mdec                 Sony PlayStation MDEC (Motion DECoder)
 D.V.L. mimic                Mimic
 DEVIL. mjpeg                Motion JPEG
 D.VIL. mjpegb               Apple MJPEG-B
 D.V.L. mmvideo              American Laser Games MM Video
 D.V.L. motionpixels         Motion Pixels video
 DEV.L. mpeg1video           MPEG-1 video (decoders: mpeg1video mpeg1_v4l2m2m )
 DEV.L. mpeg2video           MPEG-2 video (decoders: mpeg2video mpegvideo mpeg2_v4l2m2m )
 DEV.L. mpeg4                MPEG-4 part 2 (decoders: mpeg4 mpeg4_v4l2m2m ) (encoders: mpeg4 mpeg4_v4l2m2m )
 ..V.L. mpegvideo_xvmc       MPEG-1/2 video XvMC (X-Video Motion Compensation)
 D.V.L. msa1                 MS ATC Screen
 D.VI.S mscc                 Mandsoft Screen Capture Codec
 D.V.L. msmpeg4v1            MPEG-4 part 2 Microsoft variant version 1
 DEV.L. msmpeg4v2            MPEG-4 part 2 Microsoft variant version 2
 DEV.L. msmpeg4v3            MPEG-4 part 2 Microsoft variant version 3 (decoders: msmpeg4 ) (encoders: msmpeg4 )
 D.V..S msrle                Microsoft RLE
 D.V.L. mss1                 MS Screen 1
 D.VIL. mss2                 MS Windows Media Video V9 Screen
 DEV.L. msvideo1             Microsoft Video 1
 D.VI.S mszh                 LCL (LossLess Codec Library) MSZH
 D.V.L. mts2                 MS Expression Encoder Screen
 D.VIL. mvc1                 Silicon Graphics Motion Video Compressor 1
 D.VIL. mvc2                 Silicon Graphics Motion Video Compressor 2
 D.V.L. mxpeg                Mobotix MxPEG video
 D.V.L. nuv                  NuppelVideo/RTJPEG
 D.V.L. paf_video            Amazing Studio Packed Animation File Video
 DEVI.S pam                  PAM (Portable AnyMap) image
 DEVI.S pbm                  PBM (Portable BitMap) image
 DEVI.S pcx                  PC Paintbrush PCX image
 DEVI.S pgm                  PGM (Portable GrayMap) image
 DEVI.S pgmyuv               PGMYUV (Portable GrayMap YUV) image
 D.VIL. pictor               Pictor/PC Paint
 D.VIL. pixlet               Apple Pixlet
 DEV..S png                  PNG (Portable Network Graphics) image
 DEVI.S ppm                  PPM (Portable PixelMap) image
 DEVIL. prores               Apple ProRes (iCodec Pro) (decoders: prores prores_lgpl ) (encoders: prores prores_aw prores_ks )
 D.VI.S psd                  Photoshop PSD file
 D.VIL. ptx                  V.Flash PTX image
 D.VI.S qdraw                Apple QuickDraw
 D.V.L. qpeg                 Q-team QPEG
 DEV..S qtrle                QuickTime Animation (RLE) video
 DEVI.S r10k                 AJA Kona 10-bit RGB Codec
 DEVI.S r210                 Uncompressed RGB 10-bit
 DEVI.S rawvideo             raw video
 D.VIL. rl2                  RL2 video
 DEV.L. roq                  id RoQ video (decoders: roqvideo ) (encoders: roqvideo )
 D.V.L. rpza                 QuickTime video (RPZA)
 D.V..S rscc                 innoHeim/Rsupport Screen Capture Codec
 DEV.L. rv10                 RealVideo 1.0
 DEV.L. rv20                 RealVideo 2.0
 D.V.L. rv30                 RealVideo 3.0
 D.V.L. rv40                 RealVideo 4.0
 D.V.L. sanm                 LucasArts SANM/SMUSH video
 D.V.LS scpr                 ScreenPressor
 D.V..S screenpresso         Screenpresso
 DEVI.S sgi                  SGI image
 D.VI.S sgirle               SGI RLE 8-bit
 D.VI.S sheervideo           BitJazz SheerVideo
 D.V.L. smackvideo           Smacker video (decoders: smackvid )
 D.V.L. smc                  QuickTime Graphics (SMC)
 D.V... smvjpeg              Sigmatel Motion Video
 DEV.LS snow                 Snow
 D.VIL. sp5x                 Sunplus JPEG (SP5X)
 D.VIL. speedhq              NewTek SpeedHQ
 D.VI.S srgc                 Screen Recorder Gold Codec
 DEVI.S sunrast              Sun Rasterfile image
 ..V..S svg                  Scalable Vector Graphics
 DEV.L. svq1                 Sorenson Vector Quantizer 1 / Sorenson Video 1 / SVQ1
 D.V.L. svq3                 Sorenson Vector Quantizer 3 / Sorenson Video 3 / SVQ3
 DEVI.S targa                Truevision Targa image
 D.VI.S targa_y216           Pinnacle TARGA CineWave YUV16
 D.V.L. tdsc                 TDSC
 D.V.L. tgq                  Electronic Arts TGQ video (decoders: eatgq )
 D.V.L. tgv                  Electronic Arts TGV video (decoders: eatgv )
 DEV.L. theora               Theora (encoders: libtheora )
 D.VIL. thp                  Nintendo Gamecube THP video
 D.V.L. tiertexseqvideo      Tiertex Limited SEQ video
 DEVI.S tiff                 TIFF image
 D.VIL. tmv                  8088flex TMV
 D.V.L. tqi                  Electronic Arts TQI video (decoders: eatqi )
 D.V.L. truemotion1          Duck TrueMotion 1.0
 D.V.L. truemotion2          Duck TrueMotion 2.0
 D.V.L. truemotion2rt        Duck TrueMotion 2.0 Real Time
 D.V..S tscc                 TechSmith Screen Capture Codec (decoders: camtasia )
 D.V.L. tscc2                TechSmith Screen Codec 2
 D.VIL. txd                  Renderware TXD (TeXture Dictionary) image
 D.V.L. ulti                 IBM UltiMotion (decoders: ultimotion )
 DEVI.S utvideo              Ut Video
 DEVI.S v210                 Uncompressed 4:2:2 10-bit
 D.VI.S v210x                Uncompressed 4:2:2 10-bit
 DEVI.S v308                 Uncompressed packed 4:4:4
 DEVI.S v408                 Uncompressed packed QT 4:4:4:4
 DEVI.S v410                 Uncompressed 4:4:4 10-bit
 D.V.L. vb                   Beam Software VB
 D.VI.S vble                 VBLE Lossless Codec
 D.V.L. vc1                  SMPTE VC-1 (decoders: vc1 vc1_v4l2m2m )
 D.V.L. vc1image             Windows Media Video 9 Image v2
 D.VIL. vcr1                 ATI VCR1
 D.VIL. vixl                 Miro VideoXL (decoders: xl )
 D.V.L. vmdvideo             Sierra VMD video
 D.V..S vmnc                 VMware Screen Codec / VMware Video
 D.V.L. vp3                  On2 VP3
 D.V.L. vp5                  On2 VP5
 D.V.L. vp6                  On2 VP6
 D.V.L. vp6a                 On2 VP6 (Flash version, with alpha channel)
 D.V.L. vp6f                 On2 VP6 (Flash version)
 D.V.L. vp7                  On2 VP7
 DEV.L. vp8                  On2 VP8 (decoders: vp8 vp8_v4l2m2m ) (encoders: vp8_v4l2m2m )
 D.V.L. vp9                  Google VP9 (decoders: vp9 vp9_v4l2m2m )
 D.VILS webp                 WebP
 DEV.L. wmv1                 Windows Media Video 7
 DEV.L. wmv2                 Windows Media Video 8
 D.V.L. wmv3                 Windows Media Video 9
 D.V.L. wmv3image            Windows Media Video 9 Image
 D.VIL. wnv1                 Winnov WNV1
 DEV..S wrapped_avframe      AVFrame to AVPacket passthrough
 D.V.L. ws_vqa               Westwood Studios VQA (Vector Quantized Animation) video (decoders: vqavideo )
 D.V.L. xan_wc3              Wing Commander III / Xan
 D.V.L. xan_wc4              Wing Commander IV / Xxan
 D.VI.. xbin                 eXtended BINary text
 DEVI.S xbm                  XBM (X BitMap) image
 DEVIL. xface                X-face image
 D.VI.S xpm                  XPM (X PixMap) image
 DEVI.S xwd                  XWD (X Window Dump) image
 DEVI.S y41p                 Uncompressed YUV 4:1:1 12-bit
 D.VI.S ylc                  YUY2 Lossless Codec
 D.V.L. yop                  Psygnosis YOP Video
 DEVI.S yuv4                 Uncompressed packed 4:2:0
 D.V..S zerocodec            ZeroCodec Lossless Video
 DEVI.S zlib                 LCL (LossLess Codec Library) ZLIB
 DEV..S zmbv                 Zip Motion Blocks Video
 ..A.L. 4gv                  4GV (Fourth Generation Vocoder)
 D.A.L. 8svx_exp             8SVX exponential
 D.A.L. 8svx_fib             8SVX fibonacci
 DEA.L. aac                  AAC (Advanced Audio Coding) (decoders: aac aac_fixed )
 D.A.L. aac_latm             AAC LATM (Advanced Audio Coding LATM syntax)
 DEA.L. ac3                  ATSC A/52A (AC-3) (decoders: ac3 ac3_fixed ) (encoders: ac3 ac3_fixed )
 D.A.L. adpcm_4xm            ADPCM 4X Movie
 DEA.L. adpcm_adx            SEGA CRI ADX ADPCM
 D.A.L. adpcm_afc            ADPCM Nintendo Gamecube AFC
 D.A.L. adpcm_aica           ADPCM Yamaha AICA
 D.A.L. adpcm_ct             ADPCM Creative Technology
 D.A.L. adpcm_dtk            ADPCM Nintendo Gamecube DTK
 D.A.L. adpcm_ea             ADPCM Electronic Arts
 D.A.L. adpcm_ea_maxis_xa    ADPCM Electronic Arts Maxis CDROM XA
 D.A.L. adpcm_ea_r1          ADPCM Electronic Arts R1
 D.A.L. adpcm_ea_r2          ADPCM Electronic Arts R2
 D.A.L. adpcm_ea_r3          ADPCM Electronic Arts R3
 D.A.L. adpcm_ea_xas         ADPCM Electronic Arts XAS
 DEA.L. adpcm_g722           G.722 ADPCM (decoders: g722 ) (encoders: g722 )
 DEA.L. adpcm_g726           G.726 ADPCM (decoders: g726 ) (encoders: g726 )
 DEA.L. adpcm_g726le         G.726 ADPCM little-endian (decoders: g726le ) (encoders: g726le )
 D.A.L. adpcm_ima_amv        ADPCM IMA AMV
 D.A.L. adpcm_ima_apc        ADPCM IMA CRYO APC
 D.A.L. adpcm_ima_dat4       ADPCM IMA Eurocom DAT4
 D.A.L. adpcm_ima_dk3        ADPCM IMA Duck DK3
 D.A.L. adpcm_ima_dk4        ADPCM IMA Duck DK4
 D.A.L. adpcm_ima_ea_eacs    ADPCM IMA Electronic Arts EACS
 D.A.L. adpcm_ima_ea_sead    ADPCM IMA Electronic Arts SEAD
 D.A.L. adpcm_ima_iss        ADPCM IMA Funcom ISS
 D.A.L. adpcm_ima_oki        ADPCM IMA Dialogic OKI
 DEA.L. adpcm_ima_qt         ADPCM IMA QuickTime
 D.A.L. adpcm_ima_rad        ADPCM IMA Radical
 D.A.L. adpcm_ima_smjpeg     ADPCM IMA Loki SDL MJPEG
 DEA.L. adpcm_ima_wav        ADPCM IMA WAV
 D.A.L. adpcm_ima_ws         ADPCM IMA Westwood
 DEA.L. adpcm_ms             ADPCM Microsoft
 D.A.L. adpcm_mtaf           ADPCM MTAF
 D.A.L. adpcm_psx            ADPCM Playstation
 D.A.L. adpcm_sbpro_2        ADPCM Sound Blaster Pro 2-bit
 D.A.L. adpcm_sbpro_3        ADPCM Sound Blaster Pro 2.6-bit
 D.A.L. adpcm_sbpro_4        ADPCM Sound Blaster Pro 4-bit
 DEA.L. adpcm_swf            ADPCM Shockwave Flash
 D.A.L. adpcm_thp            ADPCM Nintendo THP
 D.A.L. adpcm_thp_le         ADPCM Nintendo THP (Little-Endian)
 D.A.L. adpcm_vima           LucasArts VIMA audio
 D.A.L. adpcm_xa             ADPCM CDROM XA
 DEA.L. adpcm_yamaha         ADPCM Yamaha
 DEAI.S alac                 ALAC (Apple Lossless Audio Codec)
 D.A.L. amr_nb               AMR-NB (Adaptive Multi-Rate NarrowBand) (decoders: amrnb )
 D.A.L. amr_wb               AMR-WB (Adaptive Multi-Rate WideBand) (decoders: amrwb )
 D.A..S ape                  Monkey's Audio
 D.A.L. atrac1               ATRAC1 (Adaptive TRansform Acoustic Coding)
 D.A.L. atrac3               ATRAC3 (Adaptive TRansform Acoustic Coding 3)
 D.A..S atrac3al             ATRAC3 AL (Adaptive TRansform Acoustic Coding 3 Advanced Lossless)
 D.A.L. atrac3p              ATRAC3+ (Adaptive TRansform Acoustic Coding 3+) (decoders: atrac3plus )
 D.A..S atrac3pal            ATRAC3+ AL (Adaptive TRansform Acoustic Coding 3+ Advanced Lossless) (decoders: atrac3plusal )
 D.A.L. avc                  On2 Audio for Video Codec (decoders: on2avc )
 D.A.L. binkaudio_dct        Bink Audio (DCT)
 D.A.L. binkaudio_rdft       Bink Audio (RDFT)
 D.A.L. bmv_audio            Discworld II BMV audio
 ..A.L. celt                 Constrained Energy Lapped Transform (CELT)
 DEA.L. comfortnoise         RFC 3389 Comfort Noise
 D.A.L. cook                 Cook / Cooker / Gecko (RealAudio G2)
 D.A.L. dolby_e              Dolby E
 D.A.L. dsd_lsbf             DSD (Direct Stream Digital), least significant bit first
 D.A.L. dsd_lsbf_planar      DSD (Direct Stream Digital), least significant bit first, planar
 D.A.L. dsd_msbf             DSD (Direct Stream Digital), most significant bit first
 D.A.L. dsd_msbf_planar      DSD (Direct Stream Digital), most significant bit first, planar
 D.A.L. dsicinaudio          Delphine Software International CIN audio
 D.A.L. dss_sp               Digital Speech Standard - Standard Play mode (DSS SP)
 D.A..S dst                  DST (Direct Stream Transfer)
 DEA.LS dts                  DCA (DTS Coherent Acoustics) (decoders: dca ) (encoders: dca )
 D.A.L. dvaudio              DV audio
 DEA.L. eac3                 ATSC A/52B (AC-3, E-AC-3)
 D.A.L. evrc                 EVRC (Enhanced Variable Rate Codec)
 DEAI.S flac                 FLAC (Free Lossless Audio Codec)
 DEA.L. g723_1               G.723.1
 D.A.L. g729                 G.729
 D.A.L. gremlin_dpcm         DPCM Gremlin
 D.A.L. gsm                  GSM
 D.A.L. gsm_ms               GSM Microsoft variant
 D.A.L. iac                  IAC (Indeo Audio Coder)
 ..A.L. ilbc                 iLBC (Internet Low Bitrate Codec)
 D.A.L. imc                  IMC (Intel Music Coder)
 D.A.L. interplay_dpcm       DPCM Interplay
 D.A.L. interplayacm         Interplay ACM
 D.A.L. mace3                MACE (Macintosh Audio Compression/Expansion) 3:1
 D.A.L. mace6                MACE (Macintosh Audio Compression/Expansion) 6:1
 D.A.L. metasound            Voxware MetaSound
 DEA..S mlp                  MLP (Meridian Lossless Packing)
 D.A.L. mp1                  MP1 (MPEG audio layer 1) (decoders: mp1 mp1float )
 DEA.L. mp2                  MP2 (MPEG audio layer 2) (decoders: mp2 mp2float ) (encoders: mp2 mp2fixed )
 D.A.L. mp3                  MP3 (MPEG audio layer 3) (decoders: mp3 mp3float )
 D.A.L. mp3adu               ADU (Application Data Unit) MP3 (MPEG audio layer 3) (decoders: mp3adu mp3adufloat )
 D.A.L. mp3on4               MP3onMP4 (decoders: mp3on4 mp3on4float )
 D.A..S mp4als               MPEG-4 Audio Lossless Coding (ALS) (decoders: als )
 D.A.L. musepack7            Musepack SV7 (decoders: mpc7 )
 D.A.L. musepack8            Musepack SV8 (decoders: mpc8 )
 DEA.L. nellymoser           Nellymoser Asao
 DEA.L. opus                 Opus (Opus Interactive Audio Codec)
 D.A.L. paf_audio            Amazing Studio Packed Animation File Audio
 DEA.L. pcm_alaw             PCM A-law / G.711 A-law
 D.A..S pcm_bluray           PCM signed 16|20|24-bit big-endian for Blu-ray media
 D.A..S pcm_dvd              PCM signed 20|24-bit big-endian
 D.A..S pcm_f16le            PCM 16.8 floating point little-endian
 D.A..S pcm_f24le            PCM 24.0 floating point little-endian
 DEA..S pcm_f32be            PCM 32-bit floating point big-endian
 DEA..S pcm_f32le            PCM 32-bit floating point little-endian
 DEA..S pcm_f64be            PCM 64-bit floating point big-endian
 DEA..S pcm_f64le            PCM 64-bit floating point little-endian
 D.A..S pcm_lxf              PCM signed 20-bit little-endian planar
 DEA.L. pcm_mulaw            PCM mu-law / G.711 mu-law
 DEA..S pcm_s16be            PCM signed 16-bit big-endian
 DEA..S pcm_s16be_planar     PCM signed 16-bit big-endian planar
 DEA..S pcm_s16le            PCM signed 16-bit little-endian
 DEA..S pcm_s16le_planar     PCM signed 16-bit little-endian planar
 DEA..S pcm_s24be            PCM signed 24-bit big-endian
 DEA..S pcm_s24daud          PCM D-Cinema audio signed 24-bit
 DEA..S pcm_s24le            PCM signed 24-bit little-endian
 DEA..S pcm_s24le_planar     PCM signed 24-bit little-endian planar
 DEA..S pcm_s32be            PCM signed 32-bit big-endian
 DEA..S pcm_s32le            PCM signed 32-bit little-endian
 DEA..S pcm_s32le_planar     PCM signed 32-bit little-endian planar
 DEA..S pcm_s64be            PCM signed 64-bit big-endian
 DEA..S pcm_s64le            PCM signed 64-bit little-endian
 DEA..S pcm_s8               PCM signed 8-bit
 DEA..S pcm_s8_planar        PCM signed 8-bit planar
 DEA..S pcm_u16be            PCM unsigned 16-bit big-endian
 DEA..S pcm_u16le            PCM unsigned 16-bit little-endian
 DEA..S pcm_u24be            PCM unsigned 24-bit big-endian
 DEA..S pcm_u24le            PCM unsigned 24-bit little-endian
 DEA..S pcm_u32be            PCM unsigned 32-bit big-endian
 DEA..S pcm_u32le            PCM unsigned 32-bit little-endian
 DEA..S pcm_u8               PCM unsigned 8-bit
 D.A.L. pcm_zork             PCM Zork
 D.A.L. qcelp                QCELP / PureVoice
 D.A.L. qdm2                 QDesign Music Codec 2
 D.A.L. qdmc                 QDesign Music
 DEA.L. ra_144               RealAudio 1.0 (14.4K) (decoders: real_144 ) (encoders: real_144 )
 D.A.L. ra_288               RealAudio 2.0 (28.8K) (decoders: real_288 )
 D.A..S ralf                 RealAudio Lossless
 DEA.L. roq_dpcm             DPCM id RoQ
 DEA..S s302m                SMPTE 302M
 D.A.L. sdx2_dpcm            DPCM Squareroot-Delta-Exact
 D.A..S shorten              Shorten
 D.A.L. sipr                 RealAudio SIPR / ACELP.NET
 D.A.L. smackaudio           Smacker audio (decoders: smackaud )
 ..A.L. smv                  SMV (Selectable Mode Vocoder)
 D.A.L. sol_dpcm             DPCM Sol
 DEA... sonic                Sonic
 .EA... sonicls              Sonic lossless
 ..A.L. speex                Speex
 D.AI.S tak                  TAK (Tom's lossless Audio Kompressor)
 DEA..S truehd               TrueHD
 D.A.L. truespeech           DSP Group TrueSpeech
 DEAI.S tta                  TTA (True Audio)
 D.A.L. twinvq               VQF TwinVQ
 D.A.L. vmdaudio             Sierra VMD audio
 DEA.L. vorbis               Vorbis (decoders: vorbis libvorbis ) (encoders: vorbis libvorbis )
 ..A.L. voxware              Voxware RT29 Metasound
 D.A... wavesynth            Wave synthesis pseudo-codec
 DEAILS wavpack              WavPack
 D.A.L. westwood_snd1        Westwood Audio (SND1) (decoders: ws_snd1 )
 D.A..S wmalossless          Windows Media Audio Lossless
 D.A.L. wmapro               Windows Media Audio 9 Professional
 DEA.L. wmav1                Windows Media Audio 1
 DEA.L. wmav2                Windows Media Audio 2
 D.A.L. wmavoice             Windows Media Audio Voice
 D.A.L. xan_dpcm             DPCM Xan
 D.A.L. xma1                 Xbox Media Audio 1
 D.A.L. xma2                 Xbox Media Audio 2
 ..D... bin_data             binary data
 ..D... dvd_nav_packet       DVD Nav packet
 ..D... klv                  SMPTE 336M Key-Length-Value (KLV) metadata
 ..D... otf                  OpenType font
 ..D... scte_35              SCTE 35 Message Queue
 ..D... timed_id3            timed ID3 metadata
 ..D... ttf                  TrueType font
 DES... ass                  ASS (Advanced SSA) subtitle (decoders: ssa ass ) (encoders: ssa ass )
 DES... dvb_subtitle         DVB subtitles (decoders: dvbsub ) (encoders: dvbsub )
 ..S... dvb_teletext         DVB teletext
 DES... dvd_subtitle         DVD subtitles (decoders: dvdsub ) (encoders: dvdsub )
 D.S... eia_608              EIA-608 closed captions (decoders: cc_dec )
 D.S... hdmv_pgs_subtitle    HDMV Presentation Graphic Stream subtitles (decoders: pgssub )
 ..S... hdmv_text_subtitle   HDMV Text subtitle
 D.S... jacosub              JACOsub subtitle
 D.S... microdvd             MicroDVD subtitle
 DES... mov_text             MOV text
 D.S... mpl2                 MPL2 subtitle
 D.S... pjs                  PJS (Phoenix Japanimation Society) subtitle
 D.S... realtext             RealText subtitle
 D.S... sami                 SAMI subtitle
 ..S... srt                  SubRip subtitle with embedded timing
 ..S... ssa                  SSA (SubStation Alpha) subtitle
 D.S... stl                  Spruce subtitle format
 DES... subrip               SubRip subtitle (decoders: srt subrip ) (encoders: srt subrip )
 D.S... subviewer            SubViewer subtitle
 D.S... subviewer1           SubViewer v1 subtitle
 DES... text                 raw UTF-8 text
 D.S... vplayer              VPlayer subtitle
 DES... webvtt               WebVTT subtitle
 DES... xsub                 XSUB

Am I doing something wrong? Do you need more logs/info? Would you like remote access on my test XU4?

jasaw commented 5 years ago

DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_v4l2m2m )

h264_v4l2m2m encoder is definitely compiled in.

Looks like motion found the encoder, went ahead to open the encoder (avcodec_open2 function call), but the encoder doesn't like the arguments passed into it. I suspect it doesn't like something in the ctx_codec struct. Question is what doesn't it like. Can you please bump up the log level and verify that you have Using codec h264_v4l2m2m debug output from motion? It should also print out error messages from ffmpeg.

I'll also play with h264_v4l2m2m on my RPi3, see if it behaves the same way.

jasaw commented 5 years ago

@ccrisan I'm getting segfault on my RPi3 when I use h264_v4l2m2m encoder. Let's see what's tripping your h264_v4l2m2m on XU4.

Can you please post the error messages before and after the [1:ml1:Camera1] [ERR] [ENC] ffmpeg_set_codec: Could not open codec Invalid argument line? Anything after the [1:ml1] [NTC] [ENC] ffmpeg_set_codec: Using codec h264_v4l2m2m line is of interest. If you run motion in the foreground with log level 8, it should give us error messages, something like motion -n -c camera-1.conf -d 8. Enable emulate_motion on in your camera-1.conf file to trigger recording straight away.

I might need remote access to your XU4 at a later time if that's not too difficult to set up.

ccrisan commented 5 years ago

@jasaw I just dropped you an email with remote login details on my XU4. Keep us posted.

jasaw commented 5 years ago

@ccrisan Thank you for allowing remote access to your XU4. I'm taking a look at the problem now.

jasaw commented 5 years ago

@ccrisan Here's a quick update on my findings. ffmpeg failed to initialize the video encoder because we are missing the Samsung firmware file. dmesg output below:

[38618.087183] s5p-mfc 11000000.codec:: Direct firmware load for s5p-mfc-v8.fw failed with error -2 [38618.087192] s5p_mfc_load_firmware:70: Firmware is not present in the /lib/firmware directory nor compiled in kernel

I downloaded s5p-mfc-v8.fw from https://github.com/armbian/firmware/raw/master/s5p-mfc-v8.fw and placed it in /lib/firmware directory. Now ffmpeg triggers a different error in dmesg:

[40022.156171] s5p_mfc_wait_for_done_dev:34: Interrupt (dev->int_type:0, command:12) timed out [40022.156179] s5p_mfc_init_hw:244: Failed to load firmware [40022.660730] s5p_mfc_bus_reset:110: Timeout while resetting MFC. [40024.676195] s5p_mfc_wait_for_done_dev:34: Interrupt (dev->int_type:0, command:12) timed out [40024.676203] s5p_mfc_init_hw:244: Failed to load firmware [40025.180709] s5p_mfc_bus_reset:110: Timeout while resetting MFC.

I checked Memeka's 4.9.x kernel patches and suspect we need all the patches as well. Now we have 2 choices:

Upgrade to kernel 4.14. Someone reported that MFC video encoder works on 4.14 : https://forum.armbian.com/topic/8451-howto-emby-server-with-hardware-transcoding-in-xu4hc1hc2-armbian-stretch/
We pull all the patches from Memeka's repository into our own.

What's your preference? How hard is it for you to upgrade it to kernel 4.14 ?

ccrisan commented 5 years ago

I think the best way to go is to upgrade to 4.14. Hardkernel seem to maintain this branch for the XU4: https://github.com/hardkernel/linux/tree/odroidxu4-4.14.y

jasaw commented 5 years ago

@ccrisan I compiled a new XU4 image with 4.14 kernel from https://github.com/hardkernel/linux/archive/4.14.127-164.tar.gz and loaded it onto your test XU4, but I can't access it anymore. Oops, sorry. Can you please take a look at why 4.14 kernel is not working? I can't do it remotely. :-(

ccrisan commented 5 years ago

@jasaw apparently the XU4 with 4.14 does not properly honor the reboot command. The watchdog doesn't seem to do its job either.

This being said, two manual power cycles later and the XU4 is updated :) You should now be able to access it again, just like before. If you need any kind of rebooting, just let me know and I'll power-cycle it manually as soon as I get home.

We'll need to sort out this rebooting issue somehow but that's for another day.

jasaw commented 5 years ago

@ccrisan I'm still getting fundamental issue of loading the samsung firmware with 4.14 kernel. I'm not sure what's wrong with it yet. I get this in dmesg when ffmpeg attempts to use the hardware encoder.

[18991.106094] s5p_mfc_wait_for_done_dev:34: Interrupt (dev->int_type:0, command:12) timed out [18991.106102] s5p_mfc_init_hw:249: Failed to load firmware [18991.610059] s5p_mfc_bus_reset:115: Timeout while resetting MFC. [18993.634083] s5p_mfc_wait_for_done_dev:34: Interrupt (dev->int_type:0, command:12) timed out [18993.634088] s5p_mfc_init_hw:249: Failed to load firmware [18994.138056] s5p_mfc_bus_reset:115: Timeout while resetting MFC.

This is the command I used to test:

cd /home/ftp/sdcard/Camera1/2019-07-15
rm test.mp4; ffmpeg -i 05-42-54.mp4 -codec:v copy -bsf:v h264_mp4toannexb -f rawvideo - | ffmpeg -loglevel debug -r 24 -i - -an  -vcodec h264_v4l2m2m -b:v 2M -profile:v 10 -pix_fmt nv21 test.mp4

and got this error from ffmpeg because the driver couldn't load the firmware.

[h264_v4l2m2m @ 0xdd5b0] Could not find a valid device

Again, I've downloaded s5p-mfc-v8.fw from armbian and placed it in /lib/firmware directory. I'm not sure which is the official channel to get the samsung firmware. On debian, it's part of the firmware-samsung package.

I've also checked the /boot/exynos5422-odroidxu4.dtb file, which is compiled from linux-custom/arch/arm/boot/dts/exynos5420.dtsi and linux-custom/arch/arm/boot/dts/exynos5422-odroidxu4.dts. Mali GPU info is definitely in the dtsi file.

Google search didn't yield a lot of results. Any idea on what else to try? What's the difference between our OS and Armbian? Presumably the firmware loads correctly on Armbian.

jasaw commented 5 years ago

@ccrisan Did you get a chance to look into the firmware loading issue? I seem to have lost remote access to your XU4.

I checked the Ambian s5p_mfc_v8 firmware against the one in linux-firmware-20190618, and they are the same. Have you tried Ambian OS?

ccrisan commented 5 years ago

@jasaw not yet. I'll try to see what's going on during this weekend.

ccrisan commented 5 years ago

I found a first issue, but we're still far away from a solution. The firmware file was an HTML that needed redirection:

[root@meye-06323001 firmware]# cat s5p-mfc-v8.fw 
<html><body>You are being <a href="https://raw.githubusercontent.com/armbian/firmware/master/s5p-mfc-v8.fw">redirected</a>.</body></html>[root@meye-06323001 firmware]

I downloaded the actual file and now dmesg tells me:

[    1.731997] s5p-mfc 11000000.codec: preallocated 16 MiB buffer for the firmware and context buffers
[    1.732112] (NULL device *): Direct firmware load for s5p-mfc-v8.fw failed with error -2
[    1.732124] s5p_mfc_load_firmware:73: Firmware is not present in the /lib/firmware directory nor compiled in kernel
[    1.732308] s5p-mfc 11000000.codec: decoder registered as /dev/video10
[    1.732455] s5p-mfc 11000000.codec: encoder registered as /dev/video11

Will get back with more findings...

jasaw commented 5 years ago

Make sure the s5p-mfc-v8.fw file is placed in /lib/firmware directory. From your dmesg, you have encoder and decoder registered as /dev/video10 and 11, but question is which encoder is that.

Try encoding something from ffmpeg command, something like this:

cd /home/ftp/sdcard/Camera1/2019-07-15
rm test.mp4; ffmpeg -i 05-42-54.mp4 -codec:v copy -bsf:v h264_mp4toannexb -f rawvideo - | ffmpeg -loglevel debug -r 24 -i - -an  -vcodec h264_v4l2m2m -b:v 2M -profile:v 10 -pix_fmt nv21 test.mp4

This will trigger the driver to load the s5p-mfc-v8 firmware again and you can check dmesg to see what's wrong this time.

ccrisan commented 5 years ago

@jasaw the XU4 is back online, running Armbian. I emailed you the details.

jasaw commented 5 years ago

@ccrisan I seem to have lost connection to your XU4 again. From the brief moment that I managed to get on, I've checked a few things. The dmesg error that you get is normal:

[ 1.732112] (NULL device *): Direct firmware load for s5p-mfc-v8.fw failed with error -2 [ 1.732124] s5p_mfc_load_firmware:73: Firmware is not present in the /lib/firmware directory

The driver tried to load the firmware but the filesystem probably has been mounted yet. When ffmpeg attempts to use the encoder later, the driver will load the firmware again.

I installed ffmpeg from apt, and found that they're running a much newer version, but not sure if that's necessary yet. I also noticed their ffmpeg version supports hevc_v4l2m2m. I'm interested in seeing what kind of performance we get with hardware hevc encoder.

ccrisan commented 5 years ago

@jasaw the XU4 goes down from time to time following the trends of the electricity around here :)

Good to hear those lines are normal. That's what we get with motionEyeOS if the proper firmware is where it should be. If you think we should upgrade ffmpeg, just let me know.

jasaw commented 5 years ago

With armbian kernel 4.14.127-odroidxu4, it managed to load the s5p-mfc-v8 firmware:

[h264_v4l2m2m @ 0x48b120] probing device /dev/video11 [h264_v4l2m2m @ 0x48b120] driver 's5p-mfc' on card 's5p-mfc-enc' [h264_v4l2m2m @ 0x48b120] Using device /dev/video11 [h264_v4l2m2m @ 0x48b120] driver 's5p-mfc' on card 's5p-mfc-enc' [h264_v4l2m2m @ 0x48b120] output : NM21 16 buffers initialized: 0960x0720, sizeimage 00691456, bytesperline 00000960 [h264_v4l2m2m @ 0x48b120] capture: H264 04 buffers initialized: 0000x0000, sizeimage 00532480, bytesperline 00532480

At the same time, I get this error on dmesg:

[ 1110.499680] vidioc_try_fmt:1070: failed to try output format [ 1110.499690] vidioc_try_fmt:1070: failed to try output format

I don't know what that error means yet.

I forced motion to use h264_v4l2m2m encoder and got a segfault when it starts recording. I haven't got time to track it down yet.

Just for the record, armbian is running ffmpeg version 4.1.3-1. At this stage, I don't think it matters which ffmpeg version we run.

Anyway, I'm gonna get some sleep now, continue tomorrow. If you want to take a look, I've put motion source and test config in /root directory.

jasaw commented 5 years ago

I'm getting segfault in the ffmpeg library from armbian.

From the logs, motion used h264_v4l2m2m encoder, ffmpeg initialized it successfully, configured the encoder parameters (everything looks good here). As soon as motion sends a frame to it, it segfaults. I don't know why it's segfaulting. I'll need to compile our own ffmpeg library with more debug to find out where it's segfaulting.

[1:ml1] [INF] [EVT] event_new_video: Source FPS 20 [1:ml1] [NTC] [ENC] ffmpeg_set_codec: Using codec h264_v4l2m2m [1:ml1] [INF] [ENC] ffmpeg_set_quality: h264_v4l2m2m codec vbr/crf/bit_rate: 2700000 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: probing device /dev/video0 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: driver 'uvcvideo' on card 'UVC Camera (046d:081b)' [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: v4l2 output format not supported [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: probing device /dev/video21 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: driver 'exynos-gsc' on card 'exynos-gsc gscaler' [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: v4l2 capture format not supported [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: probing device /dev/video20 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: driver 'exynos-gsc' on card 'exynos-gsc gscaler' [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: v4l2 capture format not supported [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: probing device /dev/video11 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: driver 's5p-mfc' on card 's5p-mfc-enc' [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Using device /dev/video11 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: driver 's5p-mfc' on card 's5p-mfc-enc' [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: output : NM21 16 buffers initialized: 0960x0720, sizeimage 00691456, bytesperline 00000960 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: capture: H264 04 buffers initialized: 0000x0000, sizeimage 00532480, bytesperline 00532480 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder: number of B-frames = 0 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder: header mode = 0 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder: bit rate = 2700000 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder: gop size = 10 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder Context: id (27), profile (100), frame rate(0/1), number b-frames (0), gop size (10), bit rate (2700000), qmin (2), qmax (31) [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder: h264 profile = 4 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder adjusted: qmin (0), qmax (51) [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder: minimum video quantizer scale = 0 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder: maximum video quantizer scale = 51 [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Setting default whitelist 'file,crypto' [1:ml1] [NTC] [EVT] event_newfile: File of type 8 saved to: /root/motioneye/2019-07-28/14-27-48.mp4 [1:ml1] [NTC] [ALL] motion_detected: Motion detected - starting event 1 [1:ml1] [INF] [ENC] ffmpeg_encode_video: avcodec_send_frame encode Segmentation fault

jasaw commented 5 years ago

@ccrisan I've tracked the segfault down to dereferencing of a pointer that's passed from motion software. The pointer looks sensible, but don't know why it segfaults. This is going to need more investigation. I've reported the issue to motion project, hoping to get some guidance: https://github.com/Motion-Project/motion/issues/977

jasaw commented 5 years ago

@ccrisan I found out that motion is not setting up ffmpeg buffers correctly by calling av_frame_get_buffer, which caused the earlier segfault when dereferencing buf[0]->size. I fixed that in motion and now getting segfault when dereferencing buf[1]->size. This segfault is not motion's bug anymore. I get this same segfault even with ffmpeg's own example code. I've also tried various versions of ffmpeg, with and without Memeka's patch. Anyway, I've reported the problem here: https://trac.ffmpeg.org/ticket/8079#ticket Let's see what comes out of it.

ffmpeg example code from the master branch:

./encode_video /root/motioneye/test.mp4 h264_v4l2m2m [h264_v4l2m2m @ 0x14993b0] driver 'uvcvideo' on card 'UVC Camera (046d:081b)' [h264_v4l2m2m @ 0x14993b0] driver 'exynos-gsc' on card 'exynos-gsc gscaler' [h264_v4l2m2m @ 0x14993b0] driver 'exynos-gsc' on card 'exynos-gsc gscaler' [h264_v4l2m2m @ 0x14993b0] driver 's5p-mfc' on card 's5p-mfc-enc' [h264_v4l2m2m @ 0x14993b0] Using device /dev/video11 [h264_v4l2m2m @ 0x14993b0] driver 's5p-mfc' on card 's5p-mfc-enc' [h264_v4l2m2m @ 0x14993b0] Encoder does not support b-frames yet [h264_v4l2m2m @ 0x14993b0] h264 profile not found [h264_v4l2m2m @ 0x14993b0] Encoder adjusted: qmin (0), qmax (51) Send frame 0 ffmpeg: avcodec_send_frame called ffmpeg: calling codec->send_frame v4l2_m2m_enc: v4l2_send_frame called v4l2_context: ff_v4l2_context_enqueue_frame called calling ff_v4l2_buffer_avframe_to_buf v4l2_buffers: ff_v4l2_buffer_avframe_to_buf called, out ptr: 0x149b4a0 num_planes = 2 0: v4l2_bufref_to_buf 0: frame ptr = 0x149ea70 0: &frame->buf[0] = 0x149eb68 0: frame->buf[0] = 0x149ec70 0: frame->buf[0]->size = 156800 0: frame->buf[0]->data ptr = 0xb654a020 1: v4l2_bufref_to_buf 1: frame ptr = 0x149ea70 1: &frame->buf[1] = 0x149eb6c 1: frame->buf[1] = (nil) Segmentation fault

ffmpeg command line program from the master branch too:

ffmpeg -loglevel debug -i /dev/video0 -c:v h264_v4l2m2m -b:v 2M -profile:v 66 /root/motioneye/test.mp4 ffmpeg version N-94529-g3a557c5d88 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 8 (Debian 8.3.0-6) configuration: libavutil 56. 33.100 / 56. 33.100 libavcodec 58. 55.100 / 58. 55.100 libavformat 58. 30.100 / 58. 30.100 libavdevice 58. 9.100 / 58. 9.100 libavfilter 7. 58.100 / 7. 58.100 libswscale 5. 6.100 / 5. 6.100 libswresample 3. 6.100 / 3. 6.100 Splitting the commandline. Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'. Reading option '-i' ... matched as input url with argument '/dev/video0'. Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'h264_v4l2m2m'. Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '2M'. Reading option '-profile:v' ... matched as option 'profile' (set profile) with argument '66'. Reading option '/root/motioneye/test.mp4' ... matched as output url. Finished splitting the commandline. Parsing a group of options: global . Applying option loglevel (set logging level) with argument debug. Successfully parsed a group of options. Parsing a group of options: input url /dev/video0. Successfully parsed a group of options. Opening an input file: /dev/video0. [video4linux2,v4l2 @ 0x1853370] fd:3 capabilities:84200001 [video4linux2,v4l2 @ 0x1853370] Current input_channel: 0, input_name: Camera 1, input_std: 0 [video4linux2,v4l2 @ 0x1853370] Querying the device for the current frame size [video4linux2,v4l2 @ 0x1853370] Setting frame size to 960x720 [video4linux2,v4l2 @ 0x1853370] The V4L2 driver changed the pixel format from 0x32315559 to 0x56595559 [video4linux2,v4l2 @ 0x1853370] Trying to set codec:rawvideo pix_fmt:yuv420p [video4linux2,v4l2 @ 0x1853370] The V4L2 driver changed the pixel format from 0x32315559 to 0x56595559 [video4linux2,v4l2 @ 0x1853370] Trying to set codec:rawvideo pix_fmt:yuv420p [video4linux2,v4l2 @ 0x1853370] The V4L2 driver changed the pixel format from 0x32315659 to 0x56595559 [video4linux2,v4l2 @ 0x1853370] Trying to set codec:rawvideo pix_fmt:yuv422p [video4linux2,v4l2 @ 0x1853370] The V4L2 driver changed the pixel format from 0x50323234 to 0x56595559 [video4linux2,v4l2 @ 0x1853370] Trying to set codec:rawvideo pix_fmt:yuyv422 [video4linux2,v4l2 @ 0x1853370] All info found Input #0, video4linux2,v4l2, from '/dev/video0': Duration: N/A, start: 120402.669535, bitrate: 110592 kb/s Stream #0:0, 1, 1/1000000: Video: rawvideo, 1 reference frame (YUY2 / 0x32595559), yuyv422, 960x720, 0/1, 110592 kb/s, 10 fps, 10 tbr, 1000k tbn, 1000k tbc Successfully opened the file. Parsing a group of options: output url /root/motioneye/test.mp4. Applying option c:v (codec name) with argument h264_v4l2m2m. Applying option b:v (video bitrate (please use -b:v)) with argument 2M. Applying option profile:v (set profile) with argument 66. Successfully parsed a group of options. Opening an output file: /root/motioneye/test.mp4. File '/root/motioneye/test.mp4' already exists. Overwrite ? [y/N] Y [file @ 0x1858430] Setting default whitelist 'file,crypto' Successfully opened the file. Stream mapping: Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_v4l2m2m)) Press [q] to stop, [?] for help cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [rawvideo @ 0x1855ac0] PACKET SIZE: 1382400, STRIDE: 1920 detected 8 logical cores [graph 0 input from stream 0:0 @ 0x1859f60] Setting 'video_size' to value '960x720' [graph 0 input from stream 0:0 @ 0x1859f60] Setting 'pix_fmt' to value '1' [graph 0 input from stream 0:0 @ 0x1859f60] Setting 'time_base' to value '1/1000000' [graph 0 input from stream 0:0 @ 0x1859f60] Setting 'pixel_aspect' to value '0/1' [graph 0 input from stream 0:0 @ 0x1859f60] Setting 'sws_param' to value 'flags=2' [graph 0 input from stream 0:0 @ 0x1859f60] Setting 'frame_rate' to value '10/1' [graph 0 input from stream 0:0 @ 0x1859f60] w:960 h:720 pixfmt:yuyv422 tb:1/1000000 fr:10/1 sar:0/1 sws_param:flags=2 [AVFilterGraph @ 0x1859580] query_formats: 3 queried, 2 merged, 0 already done, 0 delayed [h264_v4l2m2m @ 0x1857460] probing device /dev/video0 [h264_v4l2m2m @ 0x1857460] driver 'uvcvideo' on card 'UVC Camera (046d:081b)' [h264_v4l2m2m @ 0x1857460] v4l2 output format not supported [h264_v4l2m2m @ 0x1857460] probing device /dev/video21 [h264_v4l2m2m @ 0x1857460] driver 'exynos-gsc' on card 'exynos-gsc gscaler' [h264_v4l2m2m @ 0x1857460] v4l2 capture format not supported [h264_v4l2m2m @ 0x1857460] probing device /dev/video20 [h264_v4l2m2m @ 0x1857460] driver 'exynos-gsc' on card 'exynos-gsc gscaler' [h264_v4l2m2m @ 0x1857460] v4l2 capture format not supported [h264_v4l2m2m @ 0x1857460] probing device /dev/video11 [h264_v4l2m2m @ 0x1857460] driver 's5p-mfc' on card 's5p-mfc-enc' [h264_v4l2m2m @ 0x1857460] Using device /dev/video11 [h264_v4l2m2m @ 0x1857460] driver 's5p-mfc' on card 's5p-mfc-enc' [h264_v4l2m2m @ 0x1857460] output : NM21 16 buffers initialized: 0960x0720, sizeimage 00691456, bytesperline 00000960 [h264_v4l2m2m @ 0x1857460] capture: H264 04 buffers initialized: 0000x0000, sizeimage 00532480, bytesperline 00532480 [h264_v4l2m2m @ 0x1857460] Encoder: number of B-frames = 0 [h264_v4l2m2m @ 0x1857460] Encoder: header mode = 0 [h264_v4l2m2m @ 0x1857460] Encoder: bit rate = 2000000 [h264_v4l2m2m @ 0x1857460] Encoder: gop size = 12 [h264_v4l2m2m @ 0x1857460] Encoder Context: id (27), profile (66), frame rate(10/1), number b-frames (0), gop size (12), bit rate (2000000), qmin (2), qmax (31) [h264_v4l2m2m @ 0x1857460] Encoder: h264 profile = 0 [h264_v4l2m2m @ 0x1857460] Encoder adjusted: qmin (0), qmax (51) [h264_v4l2m2m @ 0x1857460] Encoder: minimum video quantizer scale = 0 [h264_v4l2m2m @ 0x1857460] Encoder: maximum video quantizer scale = 51 Output #0, mp4, to '/root/motioneye/test.mp4': Metadata: encoder : Lavf58.30.100 Stream #0:0, 0, 1/10240: Video: h264 (h264_v4l2m2m) (Baseline), 1 reference frame (avc1 / 0x31637661), yuyv422, 960x720, 0/1, q=2-31, 2000 kb/s, 10 fps, 10240 tbn, 10 tbc Metadata: encoder : Lavc58.55.100 h264_v4l2m2m Clipping frame in rate conversion by 0.000008 ffmpeg: avcodec_send_frame called ffmpeg: calling codec->send_frame v4l2_m2m_enc: v4l2_send_frame called v4l2_context: ff_v4l2_context_enqueue_frame called calling ff_v4l2_buffer_avframe_to_buf v4l2_buffers: ff_v4l2_buffer_avframe_to_buf called, out ptr: 0x185c860 num_planes = 2 0: v4l2_bufref_to_buf 0: frame ptr = 0x1860870 0: &frame->buf[0] = 0x1860968 0: frame->buf[0] = 0x1859560 0: frame->buf[0]->size = 1382400 0: frame->buf[0]->data ptr = 0xb6c78000 1: v4l2_bufref_to_buf 1: frame ptr = 0x1860870 1: &frame->buf[1] = 0x186096c 1: frame->buf[1] = (nil) Segmentation fault

ccrisan commented 5 years ago

@jasaw thanks for keeping us up to date with the progress. If you feel like we need to provide a patch to ffmpeg as well, we could do that :)

jasaw commented 5 years ago

Just a quick update.

Turns out that the samsung v4l2m2m driver only accepts multi-planar image data, but av_frame_get_buffer from ffmpeg only allocates one buffer rather than 2, which causes the segfault. I hacked the av_frame_get_buffer function to always allocate 2 buffers and that got rid of the segfault for now. Obviously this is not the correct way of fixing it, but let's stop it from crashing first. I tried recording video from motion, but only got a green screen. I don't know what's wrong yet, maybe video format or YUV data not packed correctly.

FFmpeg's encode_video example hasn't been updated for a while, and it really is not a very good example. I'm hacking it so that I can get something working in a small code base, but I'm getting segfault when writing encoded frame to output file via av_write_frame call.

That's where I'm up to.

jasaw commented 5 years ago

@ccrisan I managed to record successfully using h264_v4l2m2m encoder, but I can't verify whether the CrCb colour channels are correct (it's almost pitch black at your place). Again, I hacked ffmpeg and motion to get this to work. Once we can verify the encoded video is correct, we'll have to implement the fix properly.

jasaw commented 5 years ago

@ccrisan Currently I modified motion to always allocate multi-planar image buffers for ffmpeg to pass to the Samsung driver, which stops the segfault. The consequence of this is one additional memcpy to get the raw image across to this multi-planar image buffer. I'm also converting YUV420 format to NV21 format in motion by interleaving the U and V bytes. Now there's some sunlight at your place, I can verify that the h264_v4l2m2m is indeed working. If you want to play around with it, you can try running /root/motion-release-4.2.2/motion -c /root/camera-temp.conf -d 9 -n, which will record constantly to /root/motioneye directory. Be warned that it is VERY verbose at the moment because I've sprinkled debug print everywhere. It's just a proof of concept. :-)

We should probably look at how to clean this up. I'm not sure how to fix it properly yet.

Motion should automatically detect the format supported by the encoder, then convert YUV420 to NV21 via an ffmpeg function.
I'm not sure where to fix the multi-planar causing segfault issue. FFmpeg should probably be fixed to NOT segfault, but I'm not putting my hopes on ffmpeg developers to do anything about my bug report. That's why I'm working around it in motion by always allocating the buffers for multi-planar.

Any ideas are welcomed.

jasaw commented 5 years ago

There seems to be another problem. The encoded output seems to be missing a keyframe. When I run ffprobe on the output file, I get

st: 0 edit list: 1 Missing key frame while searching for timestamp: 0

This causes some video players to be very unhappy. Plays fine on some.

I don't know how to fix it.

jasaw commented 5 years ago

The missing key frame at timestamp 0 appears to be caused by the h264_v4l2m2m encoder outputting some header as the first frame. The header bytes look like this:

[ 00 00 00 01 67 64 00 0a ac b4 0b 04 b2 00 00 00 01 68 ce 01 9e 0c ]

I don't know what they mean yet, but I suspect ffmpeg setting HEADER_MODE_SEPARATE in the encoder may have something to do with it.

jasaw commented 5 years ago

HEADER_MODE_SEPARATE is indeed the cause of the missing key frame problem. In this separate header mode, the encoder first spits out the stream header (which contains NAL units) as the first packet with pts 0, then spits out the first encoded frame (which is also a key frame).

I see 2 solutions to this:

Change HEADER_MODE_SEPARATE to HEADER_MODE_JOINED_WITH_1ST_FRAME. I tried this, but ffmpeg locks up waiting forever for data from the encoder. I added a 200ms timeout in ffmpeg and managed to output a properly formed media file.
Figure out a way to detect which packet is header, which is image, and properly write the header information to the output media file.

jasaw commented 5 years ago

@ccrisan I finally got h264_v4l2m2m working on your XU4 running Armbian. I had to make slight changes to ffmpeg to get "joined with 1st frame" header mode to work. On motion side, I had to make more changes, basically handle h264_v4l2m2m a bit differently in a few places. Can you please have a play with it, let me know if it actually works (I only tested with encoding static frames since I'm developing remotely)? /root/motion-release-4.2.2/motion -c /root/camera-temp.conf -d 9 -n should get you going. Recorded videos are stored in /root/motioneye directory. At some point, we should let this system run for 1 week to check stability. Let me know when you want to run this stability test and I won't mess with the system.

For your reference, all my work is here: https://github.com/jasaw/motioneyeos/tree/xu4-hw-h264

This leaves us with one last problem: Unable to load the s5p-mfc-v8 firmware on MotionEyeOS, but works fine on Armbian.

Just a few notes: All this stuff is highly experimental, and I haven't checked whether my changes to motion broke anything on other platforms. I also need to upstream the motion patches, which means our patches now are nowhere near final.

ccrisan commented 5 years ago

@jasaw let's start the long test right away. I don't play with my XU4 these days so just let it do its stuff. There's one single problem: electricity may go off every now and then so you'll probably need to set things to start up automatically (probably using /etc/rc.local or something), just in case.

This leaves us with one last problem: Unable to load the s5p-mfc-v8 firmware on MotionEyeOS, but works fine on Armbian.

I'll take care of that.

Just a few notes: All this stuff is highly experimental, and I haven't checked whether my changes to motion broke anything on other platforms. I also need to upstream the motion patches, which means our patches now are nowhere near final.

Let's apply these patches only on the XU4 build, at least for now. This way, we reduce the possible negative impact to the 0.1% of the users (that use the XU4). I am one of them and I'll make sure to report back any issues.

jasaw commented 5 years ago

@ccrisan I have installed motioneye on your XU4 and it should start up automatically. It is configured to save movies to /var/lib/motioneye on motion trigger and remove files after 1 day. If you are physically near the camera, can you please trigger some motion every once in a while to test?

ccrisan commented 5 years ago

@jasaw will see what I can do about motion. Can you configure the threshold to a low value so that motion is often detected even though nothing really moves around?

jasaw commented 5 years ago

@ccrisan I've set the motion threshold quite low, just not sure how low I should go. I don't want to fill up the storage until the device fails to boot. It has recorded a few videos so far, and looks pretty good to me.

I will send it emulate motion every now and then to trigger recording.

jasaw commented 5 years ago

@ccrisan I've just discovered another issue with this encoder. The encoded video doesn't skip the "motion gap" sections. I suspect this encoder is ignoring the PTS in the input frame. I'll think about a workaround for it. Probably mess with the encoded PTS field.

jasaw commented 5 years ago

Turns out the static images recorded during "motion gap" is caused by a bug in motion. It has nothing to do with this encoder. The encoder does work properly after I fixed this bug in motion. See https://github.com/Motion-Project/motion/pull/997

jasaw commented 5 years ago

@ccrisan There's one small issue left with the h264_v4l2m2m encoder: There's one occasion where there was a big change in the frames that caused blocky video, but the encoder did not recover from the blockiness. Usually it recovers, but in this one instance, it did not. I think this is deep within samsung firmware which I won't be able to fix.

Other than that one issue, it has been running very well from what I can see.

ccrisan commented 5 years ago

@jasaw what does it actually do when that happens? Does it block and the whole video encoding process is dead until reboot?

jasaw commented 5 years ago

It doesn't block the process. You just get a lower quality video, looks as though bit rate is too low.

ccrisan commented 5 years ago

@jasaw do we have a workaround for this issue? Like detecting it somehow and rebooting the system?

jasaw commented 5 years ago

@ccrisan There's no need to reboot the system. The encoder recovers the next time we encode a video.

ccrisan commented 5 years ago

@jasaw that sounds reasonable enough. So the only real issue left here is updating motionEyeOS to include and load the correct firmware?

jasaw commented 5 years ago

@ccrisan This video /root/motioneye/19-41-49.mp4 on your XU4 that shows the bit rate problem in the samsung encoder. The video started out at perfect quality, but after the camera moved, the video became blocky. Motion sets the GOP to 1 second, and forcefully set the keyframe once per second, but I don't see keyframe in the encoded video. If there's a keyframe after camera move, the video would have recovered from the blockiness. I'll look into ffmpeg code to see if keyframe is ignored for v4l2 encoders.

Other than this, yes, only issue left is the include and load firmware in motionEyeOS.

jasaw commented 5 years ago

I used ffprobe to print every frame in the encoded video. Turns out that every frame is a keyframe . Setting the v4l2 header mode back to separate causes it to output I and P frames correctly. Can you please attach the camera back for me to do some more testing?

ccrisan commented 5 years ago

@jasaw sure! My apologies, I needed the USB camera for other tests but I forgot to plug it back in. It's now back in place.

motioneye-project / motioneyeos