fetzerch
commented
8 years ago @garbear: I've spent some hours to look into the bug (especially now that also people in the forums have confirmed crashes in snes9x and snes9x-next). Unfortunately I don't know ffmpeg or the video player and techniques behind it good enough to figure out what is wrong. Could you please have a look if you have time? Or is there any ffmpeg expert in the team who could help?
CDVDCodecUtils::AllocatePicture:57 allocates a data buffer on the heap to store video frames in:
int totalsize = (iWidth * iHeight) + size * 2;
uint8_t* data = new uint8_t[totalsize];For a Super Mario World rom, I get the following values: height=224, width=256, totalsize=86016, format=AV_PIX_FMT_RGB565LE.
In ColorspaceConversion (passing on to ffmpeg) some 'magic' happens and writes 1byte too much into that buffer and breaks the heap memory (which leads into the crash on the next malloc/free call).
I can reproduce the same crash (with same valgrind log) also in snes9x-next.
Any help is much appreciated. Thanks, Christian
garbear
I've seen some crashes with heap corruption warnings when using the Snes9x core. The crash seems to happen either when starting a game or when stopping it (due to a detected memory corruption in the next call to malloc or free). It happens with some games and is always reproducible.
Valgrind log: