search

shizunge / endlessh-go

A golang implementation of endlessh exporting Prometheus metrics, visualized by a Grafana dashboard.
GNU General Public License v3.0
977 stars 31 forks source link

feature request: Add geo info to log #94

Closed FoxxMD closed 8 months ago

FoxxMD commented 8 months ago

I'm working on an enhanced crowdsec collection for endlessh-go as well as a basic notifier to discord. Neither of these can take advantage of prometheus exports directly and instead need to use the log file generated to parse events. It would be helpful if the geo information parsed for prometheus metrics was included in the log lines (on ACCEPT) so that I don't have to make another call to ip-api to get the same information again.

shizunge commented 8 months ago

  1. Short answer is No.

    If you read the introduction, one reason I wrote endlessh-go is I don't want to parse unstructured logs. So adding more information to log to be parsed later defeats the purpose of this program.

    I hope you can set the expectation that these log formats could change anytime and it will break your parser.

  2. We may add structured logging. However we need to answer the following questions. We can keep this bug open for discussion.

    1. Shall we build a structured data ourselves then use current logging library to log them? Or use a new structured logging library?
    2. If we use a new library, shall we keep current logging library https://github.com/golang/glog as well? Does it worth to maintain two logging libraries in one program?
    3. If we remove current logging library, it would change the CLI interface, and break backward compatible. Does it worth the trouble to everyone? What is the rollout plan?

    Anyway it won't be done very soon.

  3. Grafana can send alerts as well https://grafana.com/docs/grafana/latest/alerting/

    Maybe you can build something like prometheus->grafana->discord or prometheus->grafana->crowdsec->discord?

FoxxMD commented 8 months ago

Appreciate the response and understand the reluctance to change the existing log format.

I went ahead and built the parser and implemented the geolocation stuff myself. For anyone else viewing this you can use endless-notify to get notifications with geolocation data to discord, ntfy, and gotify for real-time events from endlessh-go or the original endlessh. And I'll have a generic webhook and/or structure log output option available soon.