rr- / szurubooru

Image board engine, Danbooru-style.
GNU General Public License v3.0
678 stars 174 forks source link

Uploading certain files freezes host and/or causes szuru server to crash #673

Open squarerootof1 opened 1 month ago

squarerootof1 commented 1 month ago

Hiya, I've been having almost constant issues when uploading certain files to my szuru instance. Most files upload just fine, but some simply just refuse to upload and have caused the host machine that szuru is running on to freeze up and become unresponsive for long periods of time and/or have caused the szuru server docker container to exit with code 137.

I haven't exactly been able to pinpoint a solid pattern regarding what files cause my machine to freeze. Some were just the result of the image itself being semi-corrupted and uploaded just fine after a quick re-export with mspaint or photoshop. One image in particular--a 10000x10000px png with a 8192x8192px jpg version--has been giving me constant issues even after re-exporting multiple times in different file formats and I have yet to successfully upload it to my szuru instance.

And when the host machine decides not to freeze for long periods of time, szuru-toolkit (which I use to bulk upload my collection) spits out an error stating "An error occured during the similarity check: Expecting value: line 1 column 1 (char 0). Skipping post..." and the web interface simply states "Unsuccessful HTTP response" after which (as mentioned) causes the szuru docker container to exit.

I've had someone else try uploading some problem images to their own szuru instances and they've said it's uploaded with zero issues whatsoever. Part of me feels that the machine I'm using (an Oracle Cloud Standard.E2.1.micro compute instance, since it's free and, despite the notably low-end specifications, has been running szuru perfectly fine otherwise) might be part of the culprit here but I'm honestly not sure.

I've tried setting the "allow_broken_uploads" flag in the server/config.yaml to true, I've tried limiting the cpu usage on the szuru server docker container itself, yet I keep having this issue constantly to the point where it's genuinely extremely frustrating to work with. Any advice or insight would be greatly appreciated.

Additional info/clarification and config screenshots can be provided if needed.

G1org1owo commented 4 weeks ago

Have you tried manually uploading the file through the web interface? If so, can you tell us what error did you get?

I had a similar problem when trying to upload "larger" files (over 500MB iirc) and in the end I fixed the problem by allocating more swap space, as it turned out the szuru backend duplicated the file in memory and just kept running out of RAM.

Though if you have tried with a jpg version as well it might be something completely unrelated so I don't have any clue where to look for without the szuru error.