cyclestreets / cyclestreets-r

An R interface to cyclestreets.net APIs
https://rpackage.cyclestreets.net/
GNU General Public License v3.0
27 stars 7 forks source link

Batch routes: Implement new successThreshold parameter #86

Closed mvl22 closed 8 months ago

mvl22 commented 8 months ago

The batch routes API has had 99% set as the fixed-value successful threshold for the percentage of routes successfully planned. If not met, the job gets stuck in finalisation state (which we plan to change imminently). The threshold exists to avoid the scenario whereby client uses end up consuming junky data which contains large numbers of failures due to e.g. being in an unroutable area.

There is now a new successThreshold parameter which is now set on a per-job basis, defaulting to 99. This can be set via the API alongside other parameters. It can also be set in the UI. Note it is an int, not float.

https://www.cyclestreets.net/api/v2/batchroutes.createjob/#parametersoptional

This new successThreshold parameter should be implemented in the batch function for this package.

We plan very shortly to enabling a failure below the threshold as a hard fail which will not commission the data files. This would likely result in a new failure status rather than finalising/complete. At present the logic in the code is a little inconsistent due to competing requests from users over the years, whereby the status is stuck on finalisation, the files are not pushed (e.g. to S3 if enabled), but are still written, which the API then discovers.