algorand / conduit

Algorand's data pipeline framework.
MIT License
37 stars 26 forks source link

Log Rotation: document or provide solution #134

Open tzaffi opened 1 year ago

tzaffi commented 1 year ago

Problem

After a catchup on mainnet with log level INFO, the conduit log is more than 12GB. This is large enough that some sort of log rotation strategy is called for.

Solution

We could either provide our own solution using a go library, or document how this could be done on various system. For example, on linux one could use logrotate.

An argument for providing our own solution is platform independence, but a counter-argument is that this is non-standard.

Dependencies

Urgency

Low - I'm not sure how big of an issue this in the community