grafana / tempo

Grafana Tempo is a high volume, minimal dependency distributed tracing backend.
https://grafana.com/oss/tempo/
GNU Affero General Public License v3.0
3.93k stars 510 forks source link

Logger customization to avoid parse error #3871

Open icemanDD opened 2 months ago

icemanDD commented 2 months ago

Is your feature request related to a problem? Please describe.

https://github.com/grafana/dskit/issues/536

We are getting this error when try exporter logs from Tempo as json format:

{"index"=>{"status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [ts] of type [float] in document with id 'wsMnW5AB6IW1ExbRjrjx'. Preview of field's value: '2024-06-27T19:23:52.997740845Z'", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: \"2024-06-27T19:23:52.997740845Z\""}}}}

and i notice this in dskit server config

Log                          gokit_log.Logger `yaml:"-"`

Is it possible to configure the logger in yaml to remove ts or rename ts to timestamp to match our logging system schema?

Describe the solution you'd like We can customize server logger to add/edit/remove default logged fields

Describe alternatives you've considered N/A

Additional context N/A

joe-elliott commented 2 months ago

The issue is this line here? You would like to change the field name?

https://github.com/grafana/tempo/blob/main/pkg/util/log/log.go#L28

It seems we call that init method in main. If you would like to suggest some config options we'd likely take a PR to make some of this configurable.

icemanDD commented 2 months ago

I am thinking adding some config like log_fields which will accept a list of key value pairs to override what is already configured in the logger

joe-elliott commented 1 month ago

we would definitely take a PR to configure these fields if it helps you

icemanDD commented 1 month ago

thank you, this will help a lot!