pelias / openaddresses

Pelias import pipeline for OpenAddresses.
MIT License
51 stars 43 forks source link

download: suppress progress indicator for aws s3 cp command #487

Closed missinglink closed 2 years ago

missinglink commented 3 years ago

as discussed in https://github.com/pelias/docker/issues/266 the aws s3 cp command outputs regular progress information to stdout.

since we're using child_process.exec the stdout is buffered until the process exits, if the buffer (1MB by default) fills up then an error occurs.

we're not using the return value of child_process.exec (from what I could tell), so adding --only-show-errors suppresses it, while also not having to worry about conflicting/duplicate --no-progress flags provided by the user.

if we would like to return regular status reports of the file as it downloads then we would need to change child_process.exec to child_process.spawn and refactor accordingly, the latter allows customisation of the stdio config for stdout whereas the former does not.

resolves https://github.com/pelias/docker/issues/266