A broken photo breaks file processing

[umonkey]

A file was not properly uploaded, probably stopped mid way or something. It now causes the queue consumer to fail with this log:

[2024-06-01T16:51:00Z INFO  treemap::services::database::sqlite] Using SQLite database from /app/var/database.sqlite.
[2024-06-01T16:51:00Z INFO  treemap::services::file_storage::s3_storage] S3 client initialized, bucket=treemap region=fra1 endpoint=https://fra1.digitaloceanspaces.com.
[2024-06-01T16:51:00Z INFO  treemap::services::queue_consumer] Running queue consumer.
[2024-06-01T16:51:00Z DEBUG treemap::services::database::sqlite] Queue message 153536669134491666 updated.
[2024-06-01T16:51:00Z DEBUG treemap::services::queue] Message 153536669134491666 popped from queue, payload: {"command":"ResizeImage","params":{"id":153536668941553682}}
[2024-06-01T16:51:00Z INFO  treemap::services::file_storage::s3_storage] File 153536668941553682 read, 131072 bytes.
thread 'main' panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zune-jpeg-0.4.11/src/mcu.rs:209:9:
assertion `left == right` failed
  left: 362880
 right: 21966336
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This stops queue processing effectively. What needs to be fixed:

1. After reading a message from the queue, make sure it doesn't get processed until after 1 minute, to give other messages a chance on failure.
2. Handle broken files correctly, aka ignore.
3. Update supervisor config for the queue processor to delay restarts and never lock out the daemon.

[umonkey]

Example broken image.

https://github.com/umonkey/treemap/assets/16797/190cf45f-bc10-43ca-9131-743a26df90c2

[umonkey]

Apparently the is a known problem with the underlying zune library.

https://github.com/etemesi254/zune-image/issues/184