Assign FAILED status to a map request after a reasonable timeframe of remaining in PROCESSING

[rudokemper]

The MapPacker process sends a message to a mapgl-tile-renderer task worker to create a rendered offline map and updates the database post-rendering, marking it as SUCCEEDED or FAILED. However, issues such as memory or storage limits can kill a renderer container in the middle of a rendering process, leaving the map request status as PROCESSING indefinitely. To address this, we should detect such failures, set the status to FAILED, provide an error message indicating an unknown error, and write this update directly to the database.

Proposed Implementation:

I'm not sure if we want to set up a dedicated watchdog service to handle this, that seems like overkill; and I think it's better for this concern to sit with map-packer than mapgl-tile-renderer. We can have map-packer perform this check server-side every time the MapDashboard component is requested by a user, before the page is hydrated.

What should a reasonable timeframe be? I reckon 24 hours is good.

[rudokemper]

A similar need to monitor the timestamp of a map request is described in https://github.com/ConservationMetrics/map-packer/issues/11.

I might be overestimating how heavy of a lift it is to set up a watchdog service to monitor map request timestamps in the db to set them to status: FAILED or expire them; however, in the latter case, we will need a way to delete files from storage as well.

[IamJeffG]

A Container App Job can be created that runs on a schedule. It can use the same environment (so same container registery, even same Docker image, same volume mounts) as the mapgl-tile-worker.