causiq / logary

Logs and metrics are one! Professional logging, metrics and analytics for your apps.
https://docs.logary.tech/
Other
526 stars 71 forks source link

Handle implicit typing in Influx target. #233

Open mavnn opened 7 years ago

mavnn commented 7 years ago

Upgrading to the version v1.1 of InfluxDb, I'm now seeing error logs for failed writes to influx.

The error message returned from influx is:

{"error":"partial write:\nunable to parse 'event_info,service=Name.StreamProcessingManager pointName=\"ReCQ.StreamProcessing.Node.usersById\",event=eyJ2YWxpZENyZWRlbnRpYWxzIjpudWxsfQ==,key=\"1350dd8f973d41c788375db3832cc512\",event=\"Applying event to context\",value=1i 1486055837257459000': invalid boolean"}
haf commented 6 years ago

This is most likely because the shard that indexed the field as a bool with a previous event. Hard to do anything about, but we should handle it in the target.

haf commented 6 years ago
influxdb_1    | ts=2018-06-04T14:41:33.538654Z lvl=info msg="Write failed" log_id=08UiUqaW000 service=write shard=3 error="partial write: field type conflict: input field \"0\" on measurement \"EventStore\" is type string, already exists as type integer dropped=2"
haf commented 6 years ago

This will nowadays also crash the Influx target; so I'm recategorising this as a bug slated to be fixed ASAP

api_1         | [14:50:34 WRN] Exception from supervised job, restarting in 1600 ms. <Logary.Target(influx)>
api_1         | System.Exception: {"error":"partial write: field type conflict: input field \"0\" on measurement \"EventStore\" is type string, already exists as type integer dropped=2"}