pydantic / logfire

Uncomplicated Observability for Python and beyond! 🪵🔥
https://logfire.pydantic.dev/docs/
MIT License
2.16k stars 65 forks source link

Custom data retention periods #427

Open summerscope opened 1 month ago

summerscope commented 1 month ago

Summary

Currently data retention period for Logfire is one month for all users.

This design choice is important for keeping queries performant, and keeping the service affordable. However we recognise that there may be folks who need longer retention periods for a variety of reasons. If this is you, we encourage you to subscribe to this issue, and comment about your specific data retention needs.

You can also email us at engineering@pydantic.dev if you'd rather not discuss your needs publically.

samuelcolvin commented 1 month ago

👍 There's also the possibility of having different retention periods for spans vs. metrics, and having some kind of aggregations where the results are stored for longer — e.g. you store error rate, avg response time and/or e-commerce revenue aggregated by day for years.

peterlandry commented 1 month ago

Is there room for export, so we can handle long term retention in another tool?

summerscope commented 1 month ago

Is there room for export, so we can handle long term retention in another tool?

@peterlandry definitely considering raw data export - it will be for Pro plan users, not the Free tier. Feel free to email us at engineering@pydantic.dev if you'd like to tell us anything else about your use-case.