search

mpeg5 / xeve

eXtra-fast Essential Video Encoder, MPEG-5 EVC (Essential Video Coding)
Other
164 stars 39 forks source link

Segmentation fault in `xeve_pintra_create()` when running `xeve_app` or FFmpeg if compiled with `-mavx` #108

Open dbermond opened 3 months ago

dbermond commented 3 months ago

I'm getting a segmentation fault in xeve when running xeve_app or FFmpeg.

The backtrace from gdb points to xeve_pintra_create().

xeve_app

$ xeve_app -i rawvideo.y4m -o xeve.evc
XEVE: eXtra-fast Essential Video Encoder
Segmentation fault (core dumped)
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7e19a92 in xeve_pintra_create (ctx=0x7fffe3000010, complexity=0) at /usr/src/debug/xeve/xeve-0.5.0/src_base/xeve_pintra.c:541
541         ctx->fn_pintra_analyze_cu = pintra_analyze_cu;
(gdb) bt
#0  0x00007ffff7e19a92 in xeve_pintra_create (ctx=0x7fffe3000010, complexity=0) at /usr/src/debug/xeve/xeve-0.5.0/src_base/xeve_pintra.c:541
#1  0x00007ffff7def11b in xeve_platform_init (ctx=0x7fffe3000010) at /usr/src/debug/xeve/xeve-0.5.0/src_base/xeve_enc.c:769
#2  0x00007ffff7ec9938 in xevem_platform_init (ctx=0x7fffe3000010) at /usr/src/debug/xeve/xeve-0.5.0/src_main/xevem_util.c:3981
#3  0x00007ffff7de9e6f in xeve_create (cdsc=cdsc@entry=0x7fffffff24b0, err=err@entry=0x0) at /usr/src/debug/xeve/xeve-0.5.0/src_main/xevem.c:758
#4  0x00005555555574d8 in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/xeve/xeve-0.5.0/app/xeve_app.c:1117

FFmpeg

$ ffmpeg -f lavfi -i 'color=color=DodgerBlue' -c:v libxeve -t 40 -y test-out.mkv
ffmpeg version N-115229-gb0093ab8a3 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 14.1.1 (GCC) 20240507
  configuration: --prefix=/usr --enable-lto --disable-rpath --enable-gpl --enable-version3 --enable-nonfree --enable-shared --disable-static --disable-stripping --disable-htmlpages --enable-gray --enable-alsa --enable-avisynth --enable-bzlib --enable-chromaprint --enable-frei0r --enable-gcrypt --enable-gmp --enable-gnutls --enable-iconv --enable-ladspa --enable-lcms2 --enable-libaom --enable-libaribb24 --enable-libaribcaption --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcelt --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libdavs2 --enable-libdc1394 --enable-libfdk-aac --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libglslang --enable-libgme --enable-libgsm --enable-libiec61883 --enable-libilbc --enable-libjack --enable-libjxl --enable-libklvanc --enable-libkvazaar --enable-liblensfun --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopencv --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopenvino --enable-libopus --enable-libplacebo --enable-libpulse --enable-libqrencode --enable-libquirc --enable-librabbitmq --enable-librav1e --enable-librist --enable-librsvg --enable-librubberband --enable-librtmp --disable-libshaderc --enable-libshine --enable-libsmbclient --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libsvthevc --enable-libsvtvp9 --disable-libtensorflow --enable-libtesseract --enable-libtheora --disable-libtls --disable-libtorch --enable-libtwolame --enable-libuavs3d --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxevd --enable-libxeve --enable-libxavs --enable-libxavs2 --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-libxcb-shape --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-lzma --enable-decklink --disable-mbedtls --enable-libmysofa --enable-openal --enable-opencl --enable-opengl --disable-openssl --disable-pocketsphinx --enable-sndio --enable-sdl2 --enable-vapoursynth --enable-vulkan --enable-xlib --enable-zlib --enable-amf --enable-cuda-nvcc --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-libdrm --enable-libvpl --enable-libnpp --enable-nvdec --enable-nvenc --enable-omx --enable-rkmpp --enable-v4l2-m2m --enable-vaapi --enable-vdpau
  libavutil      59. 17.100 / 59. 17.100
  libavcodec     61.  5.103 / 61.  5.103
  libavformat    61.  3.103 / 61.  3.103
  libavdevice    61.  2.100 / 61.  2.100
  libavfilter    10.  2.101 / 10.  2.101
  libswscale      8.  2.100 /  8.  2.100
  libswresample   5.  2.100 /  5.  2.100
  libpostproc    58.  2.100 / 58.  2.100
Input #0, lavfi, from 'color=color=DodgerBlue':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, yuv420p, 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> evc (libxeve))
Press [q] to stop, [?] for help
Segmentation fault (core dumped)
Thread 18 "vf#0:0" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff9e400000 (LWP 24227)]
0x00007fffe308ea92 in xeve_pintra_create (ctx=0x7fff6d600010, complexity=0) at /usr/src/debug/xeve/xeve-0.5.0/src_base/xeve_pintra.c:541
541         ctx->fn_pintra_analyze_cu = pintra_analyze_cu;
(gdb) bt
#0  0x00007fffe308ea92 in xeve_pintra_create (ctx=0x7fff6d600010, complexity=0) at /usr/src/debug/xeve/xeve-0.5.0/src_base/xeve_pintra.c:541
#1  0x00007fffe306411b in xeve_platform_init (ctx=0x7fff6d600010) at /usr/src/debug/xeve/xeve-0.5.0/src_base/xeve_enc.c:769
#2  0x00007fffe313e938 in xevem_platform_init (ctx=0x7fff6d600010) at /usr/src/debug/xeve/xeve-0.5.0/src_main/xevem_util.c:3981
#3  0x00007fffe305ee6f in xeve_create (cdsc=cdsc@entry=0x555555714150, err=err@entry=0x0) at /usr/src/debug/xeve/xeve-0.5.0/src_main/xevem.c:758
#4  0x00007ffff60559f4 in libxeve_init (avctx=0x5555556f47c0) at libavcodec/libxeve.c:369
#5  0x00007ffff616bd7b in avcodec_open2 (avctx=0x5555556f47c0, codec=<optimized out>, 
    codec@entry=0x7ffff6f6f9c0 <ff_libxeve_encoder>, options=options@entry=0x555555713e68) at libavcodec/avcodec.c:326
#6  0x000055555556a9f0 in enc_open (opaque=0x555555713d80, frame=0x555555712280) at fftools/ffmpeg_enc.c:328
#7  0x0000555555589d69 in enc_open (sch=0x5555556eae80, enc=0x5555557124e0, frame=0x555555712280) at fftools/ffmpeg_sched.c:1627
#8  send_to_enc (sch=0x5555556eae80, enc=0x5555557124e0, frame=0x555555712280) at fftools/ffmpeg_sched.c:1747
#9  0x0000555555570e5b in sch_filter_send (sch=<optimized out>, fg_idx=<optimized out>, out_idx=<optimized out>, frame=0x555555712280) at fftools/ffmpeg_sched.c:2405
#10 fg_output_frame (ofp=ofp@entry=0x555555720c00, fgt=fgt@entry=0x7fff9e3f4390, frame=frame@entry=0x7fff90000900) at fftools/ffmpeg_filter.c:2420
#11 0x00005555555716ab in fg_output_step (ofp=0x555555720c00, fgt=<optimized out>, frame=0x7fff90000900) at fftools/ffmpeg_filter.c:2521
#12 read_frames (fg=fg@entry=0x555555720380, fgt=0x7fff9e3f4390, frame=0x7fff90000900) at fftools/ffmpeg_filter.c:2582
#13 0x000055555557d60c in filter_thread (arg=<optimized out>) at fftools/ffmpeg_filter.c:2999
#14 0x00005555555840df in task_wrapper (arg=0x555555720dc0) at fftools/ffmpeg_sched.c:2467
#15 0x00007ffff4ca6ded in start_thread (arg=<optimized out>) at pthread_create.c:447
#16 0x00007ffff4d2a0dc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

System information

dariusz-f commented 3 months ago

We cannot reproduce this issue. Are you build xeve by yourself or you've got from release page?

dbermond commented 3 months ago

Thank you for the reply.

I'm building it, and now I could track what is causing this.

The issue happens when building with the -mavx compiler flag, either by using it directly or by using a -march flag that enables it. For example, -march=alderlake will trigger the segmentation fault at runtime. When adding -mno-avx to CFLAGS, the problem does not happen, and both xeve_app and FFmpeg run fine.

I will update the title to reflect this.

dariusz-f commented 3 months ago

Confirmed. When build with: cmake -DCMAKE_C_FLAGS="-mavx" .. Seg fault appears when encoding with xeve_app