nestauk / dap_dv_backends

MIT License
0 stars 0 forks source link

Surround the makeRequest function with retry logic #209

Closed doogyb closed 1 year ago

doogyb commented 1 year ago

This allows all requests made to the ElasticSearch API to be "retried", if the paramaters are supplied. i.e. you can supply a retry time and retry limit to the function, so that it will wait/pause between executions. This is to fix the ES scroll bug, where the bulk request queues were being overloaded when copying an index to an s3 bucket.

fixes #208