search

quickwit-oss / quickwit

Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.
https://quickwit.io
Other
8.21k stars 336 forks source link

GCS allows only 1 write/s per key #5211

Open trinity-1686a opened 3 months ago

trinity-1686a commented 3 months ago

Per GCS documentation, we may not write an object more than once a second. For most objects, we never write more than once, but for the metastore, we may update it more frequently than that, which causes errors. We should make sure we retry in this case, and that we don't increment error metrics if the retry succeeded

trinity-1686a commented 1 month ago

retry is now implemented. Keeping the ticket opened there for a future where we may batch metastore writes