search

Azure / fluentd-plugin-mdsd

Azure Linux monitoring agent (mdsd) output plugin for fluentd
Other
27 stars 16 forks source link

parsing error in mdsd #48

Open JackQuincy opened 6 years ago

JackQuincy commented 6 years ago

I'm trying to send some logs from a glog source through to geneva and I'm getting a format error. "ProtocolHandlerJSON: Error decoding message '[mdsd.controlplane",11867,1,[5,[log",FT_STRING"],[stream",FT_STRING"],[pod",FT_STRING"],[namespace",FT_STRING"],[containerID",FT_STRING"],[FluentdIngestTimestamp",FT_TIME"]],[I0123 01:11:47.930147 1 wrap.go:42] GET /api/v1: (646.1b5}s) 200 [[hyperkube/v1.8.2 (linux/amd64) kubernetes/bdaeafa/generic-garbage-collector] 172.31.5.1:42786] ]\n","stderr","kube-apiserver-3995920765-7g09f","5a56a3b2afd0f10001c66b4f","f0675e65d26a745c0d167d1cff801c68a312b0dafb11676fa2fa6190ba79093b",[2018,0]]]' from fd 10: Invalid JSON document: Was not an array" I think it is because the log has a ] but not an opening or somthing like that. Do we need to do some encoding of the messages here? with direction I'd be happy to contribute, but I don't have much ruby experience.

jasonzio commented 6 years ago

This is a complaint from mdsd that the JSON object it received doesn't match the dynamic-json protocol mdsd expects. I'm seeing bad quoting all over the place; if you drop that object into a JSON display tool (e.g. vscode or similar) you'll see syntax complaints.

I don't know what's eating your leading quotes, but pretty much all of them are missing. When I put them in (and remove the \n from a string, which isn't really supposed to be there like that) I can see valid JSON.

JackQuincy commented 6 years ago

agreed. Which is why I put the issue here. It seems the output plugin is dropping them? or the djson socket somehow? Any pointers or thoughts on how that could happen?

JackQuincy commented 6 years ago

This is still happening a lot here is there a way to take a buffer file and push those through the output plugin and check the output?

jasonzio commented 6 years ago

It's fluentd; I'm sure there's a way to do that. shrug