search

openwisp / openwisp-monitoring

Network monitoring system written in Python and Django, designed to be extensible, programmable, scalable and easy to use by end users: once the system is configured, monitoring checks, alerts and metric collection happens automatically.
https://openwisp.io/docs/dev/monitoring/
Other
166 stars 115 forks source link

[prometheus] Metrics exporter for openwrt #100

Closed PabloCastellano closed 4 years ago

PabloCastellano commented 4 years ago

We can't use Golang's Prometheus node exporter because the binaries are too big for the openwrt devices. Fortunately @nepython has found https://stuarthowlette.me.uk/posts/prometheus-monitoring-other-os/, a node exporter written in lua with many plugins so that you can add more metrics. The code is well organized -> https://github.com/openwrt/packages/tree/openwrt-19.07/utils/prometheus-node-exporter-lua/files/usr/lib/lua/prometheus-collectors and looks like the way to go but before we arrive to this task, let's take the time to analyze all possible choices.

2stacks commented 4 years ago

Is fluentbit an option?

On Tue, May 26, 2020, 9:52 AM Pablo Castellano notifications@github.com wrote:

We can't use Golang's Prometheus node exporter because the binaries are too big for the openwrt devices. Fortunately @nepython https://github.com/nepython has found https://stuarthowlette.me.uk/posts/prometheus-monitoring-other-os/, a node exporter written in lua with many plugins so that you can add more metrics. Looks like the way to go but before we arrive to this task, let's take the time to analyze all possible choices.

  • Decide what exporter to use (or even implement our own if necessary)
  • Analyze if the metrics exposed by the existing exporters cover our current needs
  • Value how easy is to contribute back to the existing projects in case we need to implement our own metrics
  • Document how to install the exporter in an openwrt device

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openwisp/openwisp-monitoring/issues/100, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADUNEHHYEKHDS64IYTVQJETRTPCRVANCNFSM4NKKPCUA .

PabloCastellano commented 4 years ago

@2stacks From fluentbit homepage:

Fluent Bit is a Fast and Lightweight Log Processor and Forwarder for Linux, OSX and BSD family operating systems.

As I understand, Fluent bit would be helpful to stream logs but not metrics. Besides that, I have compiled fluent-bit myself and the binary weights 6MB, which is also a lot for most of openwrt devices. So I would say it's not an option at the moment, but let me know if you think I'm wrong

nepython commented 4 years ago

I checked out the docs/mailing-list and noted the below points.

Finally, I think that fluentbit is a nice tool which can be used for many things but in this case the package I have suggested is very specific for our purpose (consumes around ~10kb as informed by @PabloCastellano) and thus might be a lighter dependency to add.

@2stacks I haven't used fluent-bits. Hence, concluded this based upon what I have read, please correct me in case I miss anything and thanks for the suggestion :smile:

PabloCastellano commented 4 years ago

@nepython You are confusing metrics with logs. Check out https://www.sumologic.com/blog/logs-metrics-overview/

nemesifier commented 4 years ago

As discussed recently, we decided to proceed with Elasticsearch (see #99), hence I'm closing this issue to clean up the issue list. Thanks!