Mapbox generates QA tiles, with an internal SLA of 7 days, which means that an attempt is made to build a QA tiles tileset everyday, but if there's an issue in the build process, it is only discovered after tilesets have not been built consecutively for 7 days. On the HOT side, it is essential that we have a fresh QA tiles tileset at least once every 2-3 days. Currently, conversations are ongoing with Mapbox over the maintenance and ownership of these tilesets.
There is not complete clarity on the HOT ownership of this stack. Currently, Mapbox is owning the overall stack, and we are waiting for them to confirm if they can continue to own the stack. Based on this conversation, we want to ask if they can reduce the tileset SLA to 2 days. In case that conversation does not pan out, we want to build a mapbox monitor that notifies us, and Mapbox through us, if the QA tiles are not being updated.
Plan
Based on Mapbox's answer to the following, we can make some decisions around building a monitoring system, that notifies us if the QA tiles tileset has not been updated in the last 24 hours:
Whether they are going to continue maintaining and owning this stack
Whether we can access the source osm PBF file that is built everyday
Mapbox Monitoring bot architecture
The bot is triggered as a lambda function on CloudWatch, and is architected as in this picture:
A CloudWatch event triggers a lambda at 10 a.m IST
The lambda checks the QA tiles S3 path to see if there's a file there that has been updated in the last 24 hours.
If a file exists, the lambda does nothing
If a file does not exist, the lambda scales up the ASG by 1, which will trigger a QA tiles process that then runs to completion and pushes files onto the HOT infrastructure.
Other gotchas
If Mapbox confirms that they cannot give us access to the latest planet OSM PBFs, we will also need to build these from planet-osm everyday as part of the tileset generation process.
Next Actions
[ ] wait for Mapbox to follow up
[ ] Build the Mapbox monitor
[ ] Generate the latest OSM PBFs from planet-osm
cc/ @ramyaragupathy @smit1678 please add anything I may have missed!
Ref: https://github.com/hotosm/hot-qa-tiles/issues/2.
Issue
Mapbox generates QA tiles, with an internal SLA of 7 days, which means that an attempt is made to build a QA tiles tileset everyday, but if there's an issue in the build process, it is only discovered after tilesets have not been built consecutively for 7 days. On the HOT side, it is essential that we have a fresh QA tiles tileset at least once every 2-3 days. Currently, conversations are ongoing with Mapbox over the maintenance and ownership of these tilesets.
There is not complete clarity on the HOT ownership of this stack. Currently, Mapbox is owning the overall stack, and we are waiting for them to confirm if they can continue to own the stack. Based on this conversation, we want to ask if they can reduce the tileset SLA to 2 days. In case that conversation does not pan out, we want to build a mapbox monitor that notifies us, and Mapbox through us, if the QA tiles are not being updated.
Plan
Based on Mapbox's answer to the following, we can make some decisions around building a monitoring system, that notifies us if the QA tiles tileset has not been updated in the last 24 hours:
Mapbox Monitoring bot architecture
The bot is triggered as a lambda function on CloudWatch, and is architected as in this picture:
Other gotchas
If Mapbox confirms that they cannot give us access to the latest planet OSM PBFs, we will also need to build these from planet-osm everyday as part of the tileset generation process.
Next Actions
cc/ @ramyaragupathy @smit1678 please add anything I may have missed!