search

conveyal / r5

Developed to power Conveyal's web-based interface for scenario planning and land-use/transport accessibility analysis, R5 is our routing engine for multimodal (transit/bike/walk/car) networks with a particular focus on public transit
https://conveyal.com/learn
MIT License
272 stars 71 forks source link

Schedule creation of geotiff zip files in the background #938

Closed abyrd closed 2 months ago

abyrd commented 3 months ago

Discussion and observations at https://github.com/conveyal/r5/pull/932#issuecomment-2048894228.

It may be possible to make creation of single-cutoff geotiffs and grids faster (e.g. using NIO buffers). But for the time being it can take several minutes to produce and archive all the geotiffs for a single large regional analysis. This PR is a working proof of concept to perform this operation in the background. The process of making the zip file has been moved almost verbatim into a Task.withAction(). Usage:

> curl 'http://localhost:7070/api/regional/66179beec42a7657af4fdd2d/all' -v
HTTP/1.1 202 Accepted
"Building ZIP in background."

> curl 'http://localhost:7070/api/regional/66179beec42a7657af4fdd2d/all' -v
HTTP/1.1 202 Accepted
"ZIP is already being prepared in the background."

> curl 'http://localhost:7070/api/regional/66179beec42a7657af4fdd2d/all' -v
HTTP/1.1 200 OK
{"url":"http://localhost:7070/files/results/66179beec42a7657af4fdd2d_ALL.zip","humanName":"TEST_REGIONAL_2_4fdd2d.zip"}

Progress is reported as each file is created:

image