search

bakape / hydron

media tagger and organizer
GNU Lesser General Public License v3.0
95 stars 9 forks source link

Abort on import #77

Open bakape opened 4 years ago

bakape commented 4 years ago

When thumbnailing certain files in the middle of a group of files, hydron receives a SIGABRT and crashes with one of a number of different error messages. corrupted size vs. prev_size while consolidating malloc(): corrupted top size corrupted double-linked list free(): invalid next size (normal) Every file can be thumbnailed individually. If the import command is repeated, it will move past the point of crash but it might still crash on a later file. The order that the files are processed in seems to be important so to reproduce consistently hydron/import.go#41 should be n := 1 Both folders in the attached zip crash, I can get more examples if you like. Debian 10, thumbnailer v2.6.4, hydron and deps up to date.

test data.zip

Kamuso commented 4 years ago

I "fixed" it by setting thumbnailer.c#93 to dst->data = malloc(dst->size + 25); so av_image_get_buffer_size must be giving the wrong size for some files. I don't know where to go from here.

bakape commented 4 years ago

Why 25?

Kamuso commented 4 years ago

It's the smallest value that worked with the test data. I tried with 100 first.

Kamuso commented 3 years ago

I tried some imports running Hydron with Valgrind to see what's happening with memory. Running valgrind --read-var-info=yes --leak-check=no ./hydron import path shows invalid writes when importing the files that cause crashes. Pic related. Other files report no invalid writes. valgrind output Valgrind still reports an invalid write when using the malloc(dst->size + 25) I mentioned earlier, but malloc(dst->size + 64) is fine. All the bad images I've found have been jpgs with YUVJ420p colour encoding. Re-encoding them as png, or as jpg with YUVJ444p colour makes them thumbnail with no invalid writes, but not all YUVJ420p jpgs cause invalid writes. I tried having a look at the ffmpeg source to see where the 8 bytes are coming from but I can't figure it out. Do you have any suggestions?

Chiiruno commented 3 years ago

I believe this may be an issue with webp, as well. example_image.tar.gz wtf.log

bakape commented 3 years ago

I'll get to this eventually. Too much on my plate at the moment.

On Thu, 17 Dec 2020 at 05:22, チルノ notifications@github.com wrote:

I believe this may be an issue with webp, as well. example_image.tar.gz https://github.com/bakape/hydron/files/5706768/example_image.tar.gz wtf.log https://github.com/bakape/hydron/files/5706769/wtf.log

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bakape/hydron/issues/77#issuecomment-747179400, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB347MBSYXGVPLUPAKUTXCDSVF2QRANCNFSM4QK32OTA .

bakape commented 3 years ago

I might have fixed it in the commit above. As you can reproduce, could you try the fix? Will try to reproduce before and after commit myself a later this week.

Kamuso commented 3 years ago

Still aborts. When trying to reproduce, running valgrind --read-var-info=yes --leak-check=no ./hydron import path and looking for invalid writes is more reliable than looking for crashes, by the way.