liftbridge-io / liftbridge

Lightweight, fault-tolerant message streams.
https://liftbridge.io
Apache License 2.0
2.57k stars 107 forks source link

option for JSON logging maybe? #294

Open sigurdblueface opened 3 years ago

sigurdblueface commented 3 years ago

It seems easy to implement and it also would be handy and useful for many people who use elasticsearch/loki etc

smth like this

func NewLogger(level uint32, format string) Logger {
    l := log.New()
    l.SetLevel(log.Level(level))
    switch format {
    case "json":
        logFormatter := &log.JSONFormatter {
            TimestampFormat: "2006-01-02 15:04:05",
        }
        l.Formatter = logFormatter
    default:
        logFormatter := &log.TextFormatter{
            FullTimestamp:   true,
            TimestampFormat: "2006-01-02 15:04:05",
        }
        l.Formatter = logFormatter
    }
    return &logger{l}
}

and just an additional field in Config struct

tylertreat commented 3 years ago

Yep, agreed. Better logging is something I've had in mind. Feel free to make a PR if you'd like.

sigurdblueface commented 3 years ago

well, I'm almost done with it here, but stuck on raftLogger's Write function:

[]byte to str conversion with JSON logger results here into lots of excessive escaped double quotes and newlines:

{"level":"error","msg":"raft: failed to make requestVote RPC: target=\"{Voter b b}\" error=\"nats: timeout\"\n","time":"2020-12-17 11:51:49"}
{"level":"error","msg":"raft: failed to make requestVote RPC: target=\"{Voter b b}\" error=\"nats: timeout\"\n","time":"2020-12-17 11:51:49"}
{"level":"warning","msg":"raft: Election timeout reached, restarting election\n","time":"2020-12-17 11:51:50"}
{"level":"info","msg":"raft: entering candidate state: node=\"Node at FdFmBT7CxmSNDOtAqXEtJe [Candidate]\" term=153\n","time":"2020-12-17 11:51:50"}
^C{"level":"info","msg":"Shutting down...","time":"2020-12-17 11:51:50"}
{"level":"error","msg":"raft: failed to make requestVote RPC: target=\"{Vot

And atm idk an elegant solution to this.

probably we should parse it to json fields with an additional function i.e:

{"level":"error","msg":"raft: failed to make requestVote RPC: target=\"{Voter b b}\" error=\"nats: timeout\"\n","time":"2020-12-17 11:51:49"}

should become:

{"level":"error","msg":"failed to make requestVote RPC","target":"Voter b b","error":"nats: timeout","time":"2020-12-17 11:51:49"}

what do you think? any help or guidance would be much appreciated

tylertreat commented 3 years ago

Perhaps instead of overriding the LogOutput on the Raft config to use raftLogger, we should override the Logger. Hashicorp's logger has the ability to output as JSON (see here).