search

bitcoin-dev-project / warnet

Monitor and analyze the emergent behaviors of Bitcoin networks
https://warnet.dev
MIT License
92 stars 35 forks source link

Centralised logging / metrics for k8s #186

Open josibake opened 12 months ago

josibake commented 12 months ago

I had to rip out fork observer, fluentd for now. They might not be the best solution going forward, too! But we definitely want some sort of logging / dashboard solution

willcl-ark commented 12 months ago

I also propose to rip out fluentd for docker in this pull: https://github.com/bitcoin-dev-project/warnet/pull/173

It slows down docker too much, both on the log processing side (Ruby maxes out 1 CPU for me) and AFAIK bitcoind logs on each container going to stdout will be written to a daemon host file once (tank container), also read by fluentd, written again to another host logfile (fluentd), so doubling storage space and file read/writes. (can verify as docker logs tank_000000 will return tank 0 logs, and they will also be obn fluentd container docker logs warnet_fluentd.

Long story short we should prob just use what's best for each orchestration/container tool.

For docker I think just use default logger and the modified log_grepper fn in #173.

For k8s, which might be much larger scale, perhaps we can leverage @m3dwards expertise with an ELK stack or similar?

willcl-ark commented 9 months ago

I think @m3dwards is working on this