Segmentation Fault Converting h264 to VP9 to 4k

[jgcaruso]

Hi there

I'm trying to convert a 2 minute 30fps 4k h264 file to a 4k vp9 file using ffmpeg and I'm getting a segmentation fault around 42 seconds of video transcoded (3 minutes of processing time). The same video worked fine transcoding to 1440p video file.

ffmpeg command:

ffmpeg -i bbb-2m-4k-30fps.mp4 -y -f mp4 -s 3840x2160 -acodec aac -c:v libsvt_vp9 -movflags faststart -pix_fmt yuv420p -r 30 -g 30 -minrate 6000k -maxrate 17400k -b:v 12000k bbb-2m-svt-vp9-4k.mp4

I also tried running SvtVp9EncApp and I get the following error: command:

sudo SvtVp9EncApp -w 3840 -h 2160 -i ~/vp/bbb-2m-4k-30fps.mp4 -fps 30 -n 100 -tune 0 -q 32 -b test.bin

error:

double free or corruption (out)
Aborted

The full dump:

-------------------------------------------
SVT [version]   : SVT-VP9 Encoder Lib v0.3.0
SVT [build] : GCC 9.3.0  64 bit
LIB Build date: Oct 21 2021 15:46:52
-------------------------------------------
Number of logical cores available: 4
Number of PPCS 37
-------------------------------------------
SVT [config]: Profile [0]   Level (auto)
SVT [config]: EncoderMode / Tune                        : 9 / 0
SVT [config]: EncoderBitDepth                           : 8
SVT [config]: SourceWidth / SourceHeight                    : 3840 / 2160
SVT [config]: FrameRate / Gop Size                      : 30 / 33
SVT [config]: HierarchicalLevels / BaseLayerSwitchMode / PredStructure      : 3 / 0 / 2
SVT [config]: BRC Mode / QP                             : CQP / 32
-------------------------------------------

Encoding       100
SUMMARY --------------------------------- Channel 1  --------------------------------
Total Frames        Frame Rate      Byte Count      Bitrate
         100        30.00 fps        300000000      720000.00 kbps

Channel 1
Average Speed:      2.46 fps
Total Encoding Time:    40624 ms
Total Execution Time:   41152 ms
Average Latency:    19192 ms
Max Latency:        28294 ms
double free or corruption (out)
Aborted

I'm running this in an EC2 m6i.xlarge instance running Ubuntu CPU: Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz RAM: 16GB

The video file is a 2 minute clip of Big Buck Bunny, trimmed in Quicktime so it shouldn't be a strange format causing the issue.

Any help or direction in debugging this issue would be helpful.

[1480c1]

Can you try some more minimal reproduction cases?

ffmpeg -i bbb-2m-4k-30fps.mp4 -an -c:v libsvt_vp9 -g 30 -minrate 6000k -maxrate 17400k -b:v 12000k -f null -

# and

ffmpeg -i bbb-2m-4k-30fps.mp4 temp.yuv
SvtVp9EncApp -w 3840 -h 2160 -i temp.yuv -fps 30 -n 100 -tune 0 -q 32 -b test.bin

?

As a note, SvtVp9EncApp only supports yuv input

[jgcaruso]

As a note, SvtVp9EncApp only supports yuv input

Oh good to know!

ffmpeg -i bbb-2m-4k-30fps.mp4 -an -c:v libsvt_vp9 -g 30 -minrate 6000k -maxrate 17400k -b:v 12000k -f null -

This still seg faults.

ffmpeg -i bbb-2m-4k-30fps.mp4 temp.yuv SvtVp9EncApp -w 3840 -h 2160 -i temp.yuv -fps 30 -n 100 -tune 0 -q 32 -b test.bin

This completes with the message

Encoding       100
SUMMARY --------------------------------- Channel 1  --------------------------------
Total Frames        Frame Rate      Byte Count      Bitrate
         100        30.00 fps          4681304      11235.13 kbps

Channel 1
Average Speed:      8.17 fps
Total Encoding Time:    12234 ms
Total Execution Time:   12777 ms
Average Latency:    4389 ms
Max Latency:        5583 ms
Encoder finished

If I re-run it without -n 100 it fails with the message

free(): invalid size
Aborted (core dumped)

Full dump

-------------------------------------------
SVT [version]   : SVT-VP9 Encoder Lib v0.3.0
SVT [build] : GCC 9.3.0  64 bit
LIB Build date: Oct 21 2021 15:46:52
-------------------------------------------

[WARNING] For best speed performance, run with sudo privileges !

Number of logical cores available: 4
Number of PPCS 37
-------------------------------------------
SVT [config]: Profile [0]   Level (auto)
SVT [config]: EncoderMode / Tune                        : 9 / 0
SVT [config]: EncoderBitDepth                           : 8
SVT [config]: SourceWidth / SourceHeight                    : 3840 / 2160
SVT [config]: FrameRate / Gop Size                      : 30 / 33
SVT [config]: HierarchicalLevels / BaseLayerSwitchMode / PredStructure      : 3 / 0 / 2
SVT [config]: BRC Mode / QP                             : CQP / 32
-------------------------------------------

Encoding      2000
Average System Encoding Speed:        9.06
     3640
SUMMARY --------------------------------- Channel 1  --------------------------------
Total Frames        Frame Rate      Byte Count      Bitrate
        3640        30.00 fps        183565785      12103.24 kbps

Channel 1
Average Speed:      9.24 fps
Total Encoding Time:    394102 ms
Total Execution Time:   394627 ms
Average Latency:    6824 ms
Max Latency:        8976 ms
free(): invalid size
Aborted (core dumped)

[jgcaruso]

I also ran it again after running sudo sysctl -w kernel.sched_rt_runtime_us=1000000 and I got this error

munmap_chunk(): invalid pointer
Aborted (core dumped)

Just wondering if there is anything else I can look into, or any way to dump extra logs to find out what is going on? Is it a hardware issue where it should only be run on bare metal and this is just not meant to run on AWS?

[1480c1]

I am able to reproduce the issue even on bare metal, so I will try to see if I can debug further

as a note, I was able to do so with just

./Bin/Debug/SvtVp9EncApp -i /dev/urandom -w 3840 -h 2160 -fps 1 -n 1 -tune 0 -q 63

so no need for any specific clip nor anything other than just the 4k part.

However, the error message is not always the same (munmap_chunk(): invalid pointer vs free(): invalid size) so this is leading me to believe there's a buffer overrun or similar occurring somewhere