evalphobia / logrus_fluent

fluentd hooks for logrus
Apache License 2.0
45 stars 22 forks source link

invalid JSON message #28

Closed hellojukay closed 6 years ago

hellojukay commented 6 years ago

my code

package main

import (
    "context"
    "errors"

    "github.com/evalphobia/logrus_fluent"
    "github.com/sirupsen/logrus"
)

func main() {
    hook, err := logrus_fluent.NewWithConfig(logrus_fluent.Config{
        Host: "localhost",
        Port: 5170,
    })
    if err != nil {
        panic(err)
    }

    // set custom fire level
    hook.SetLevels([]logrus.Level{
        logrus.PanicLevel,
        logrus.ErrorLevel,
        logrus.InfoLevel,
    })

    // set static tag
    hook.SetTag("original.tag")
    // ignore field
    hook.AddIgnore("context")

    // filter func
    hook.AddFilter("error", logrus_fluent.FilterError)
    logrus.SetFormatter(&logrus.JSONFormatter{})
    logrus.AddHook(hook)
    for i := 1; i <= 1000; i++ {
        logrus.Info("Hello World")
    }
}

func logging(ctx context.Context) {
    logrus.WithFields(logrus.Fields{
        "value":   "some content...",
        "error":   errors.New("unknown error"), // this field will be applied filter function in the hook.
        "context": ctx,                         // this field will be ignored in the hook.
    }).Error("error message")
}

my log

{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}
{"level":"info","msg":"Hello World","time":"2018-09-26T11:32:44+08:00"}

fluent bit config and warn log

[jukay@mac-pro]~/Code/Go/src/github.com/hellojukay/fluent-bit-0.14.1/build/bin$ ./fluent-bit -i tcp -o stdout
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping
[2018/09/26 14:31:22] [ warn] [in_tcp] invalid JSON message, skipping

my question

Why log json error,how to fix it

evalphobia commented 6 years ago

@hellojukay Would you try it on plain fluentd?

I tried td-agent v3.1.1 on OSX and get these logs below,

2018-10-01 02:29:44 +0900 [warn]: #0 no patterns matched tag="original.tag"
2018-10-01 02:29:44 +0900 [warn]: #0 no patterns matched tag="original.tag"
hellojukay commented 6 years ago

@evalphobia i use fluent-bit not fluentd.

hellojukay commented 6 years ago

this project just used for fluentd , not for fluent bit.

evalphobia commented 6 years ago

@hellojukay I wish this library could help you out... This library use fluent/fluent-logger-golang for sending data to fluentd. If they support fluent bit, this library also can do.