Any thoughts about error handling

chronark / zod-bird

Fully typed Tinybird pipes using zod

MIT License

140 stars 13 forks source link

Open mfts opened 3 months ago

mfts commented 3 months ago

Hey @chronark

Since there have been two tinybird ingest outages within the last month. (Feb 10 and March 9)

Any thoughts how you are currently handling failed tinybird requests?

Was thinking to add error handling and retries to my implementation along the lines of this here: https://www.tinybird.co/docs/guides/ingest-from-the-events-api.html#error-handling-and-retries

How are you doing it at unkey?

chronark commented 3 months ago

It retries once already, but only on 429 errors. We can improve that, but I don't see a good solution for buffering long outages within this lib.

We're going to buffer and batch all messages in a cloudflare queue soon (similar to what we do for axiom logs already)

mfts commented 3 months ago

Nice! Would love to learn more about this when you release it for unkey