Strange FFmpeg errors

[reticivis-net]

FFmpeg seems to error out a lot of the time with strange errors that have no known fixes and are possibly ffmpeg bugs. These error messages include

• malloc(): smallbin double linked list corrupted #125 #160
• invalid fastbin entry (free) #135 #157 #163 #201
• ffmpeg: quantize.c:2045: CBR_iteration_loop: Assertion 'cod_info->part2_3_length <= targ_bits[ch]' failed. #111 (this one has to do with very loud audio levels?)
• munmap_chunk(): invalid pointer
• malloc_consolidate(): invalid chunk size #150 #149 #159 #196
• sometimes there isn't an error message, it just immediately closes. #140 #164
• malloc(): corrupted top size #158
• corrupted double-linked list https://github.com/HexCodeFFF/mediaforge/issues/151#issuecomment-1386022126 #215
• Error muxing a packet #178

[reticivis-net]

there is a strong possibility (see #149 #151 ) that the tenor gifs are causing these issues. re-encoding the tenor gif or encoding tenor mp4s as gifs could fix this.

[WR4F]

Interestingly, the issue is (at least for me) an intermittent one. The same tenor gifs are no longer causing a problem. @HexCodeFFF Did you update the bot recently with a fix or update the FFmpeg version?

As the errors suggest, some of these problems likely have to do with memory corruption and/or running out of memory.

My understanding of some of these errors:

• malloc(): smallbin double linked list corrupted - memory addresses with a data specific structure being having invalid/unexpected content
• munmap_chunk(): invalid pointer - pointers referencing invalid memory addresses which causes a crash when FFmpeg attempts to free() them (could be that the data at that pointer address is corrupted)
• malloc_consolidate(): invalid chunk size - an odd error related to attempting to perform some kind of operation on an already allocated thing in memory, seems similar to the previous error with pointers being invalid or memory being corrupted

[reticivis-net]

I didn’t update ffmpeg for a while and then I did very recently The errors are more prevalent cause the rewrite/v2 uses ffmpeg more heavily and more weirdly than before I’m just not sure what’s causing these errors. I’ll have to see if I can coax ffmpeg into throwing out a stack trace or something There’s also me using an in-memory file system inside the Linux docker that could maybe have issues? I’m not sure

[reticivis-net]

Several people have pointed to tenor gifs being the cause of these issues. Once I have the time I need to try to reliably reproduce one of these

[reticivis-net]

Ffmpeg has been updated to the latest version which is 6 I think and these errors are still occurring

[reticivis-net]

In my limited testing it seems that normal gifs, even gifs produced by MediaForge (ffmpeg) can cause this, but only gifs A possible workaround might be to convert to video, caption, then convert to gif

[Jacherr]

Most of these errors appear to be heap overflows and these kinds of issues can be caused by limited memory availability for the process.

Have you monitored memory usage while these commands are being executed? Is it possible that the process is running out of memory? This would especially be true if only being ran on gifs - especially Tenor ones, which are notoriously quite large files.

[reticivis-net]

Oh shit really? Memory on the host system is very limited and I’ve been using in memory tempfs to hopefully speed up things but I didn’t know it could cause this. I’ll turn off the in memory fs then damn thanks so much