search

m-lab / etl

M-Lab ingestion pipeline
Apache License 2.0
22 stars 7 forks source link

Task.Close should return error, and Process... should handle and/or propagate that error. #985

Open gfr10598 opened 3 years ago

gfr10598 commented 3 years ago

https://github.com/m-lab/etl/blob/5caa9cbbd394ec4f0f7cd1e82eeec6b26a21525b/task/task.go#L66-L66

Currently, these errors are dropped - not reported to Gardener.

Among these errors are GCS write errors, reported on

https://github.com/m-lab/etl/blob/5caa9cbbd394ec4f0f7cd1e82eeec6b26a21525b/storage/rowwriter.go#L168-L169

gfr10598 commented 3 years ago

There are also insufficient permission errors that should be propagated.

laiyi-ohlsen commented 3 years ago

@stephen-soltesz