I recently ran a large lightmap baking job, where some of the individual jobs took around 30 hours. This caused some problems, because Amazon's SQS service only seems to allow you to "hide" an item in the queue for up to 12h, and if you try to extend this time for longer it throws an exception. This causes the brenda-node script to terminate, and the jobs are returned to the queue, even though the Blender process is still happily churning away.
Once Blender finishes, the output file is created but not synced to s3, and the instance is not shut down. I had to do these two steps manually for my bake.
Here's a graph which shows this problem in action:
I recently ran a large lightmap baking job, where some of the individual jobs took around 30 hours. This caused some problems, because Amazon's SQS service only seems to allow you to "hide" an item in the queue for up to 12h, and if you try to extend this time for longer it throws an exception. This causes the brenda-node script to terminate, and the jobs are returned to the queue, even though the Blender process is still happily churning away.
Once Blender finishes, the output file is created but not synced to s3, and the instance is not shut down. I had to do these two steps manually for my bake.
Here's a graph which shows this problem in action: