lupinia
commented
1 year ago Upon further investigation, I'm not sure anything changed, other than the process_images command now using slightly more memory as a result of #120, just enough to put it over the edge, when it previously just ran very close to the margins. I don't see an easy fix or an obvious memory leak (beyond Python being Python), so I'll just throw more memory at the server and be done with it. The site is getting more traffic now than it has in a while anyway, and it's not like it's horrendously expensive to run, so this is fine.
I'm not entirely sure why this suddenly started happening now, but it's happening more and more frequently, and not just because I've been running a re-processing batch on the entire site for the last week. Either the process_images command is using a lot more memory than it used to, or the site in general is consuming more memory in its worker processes than it used to, thus not leaving enough for the image processing background command. And it's not consistent - sometimes this will only happen once, sometimes I'll wake up to a hundred failure emails that came in throughout the night.