search

SkyAPM / go2sky

Distributed tracing and monitor SDK in Go for Apache SkyWalking APM
https://skywalking.apache.org/
Apache License 2.0
447 stars 122 forks source link

Make span tag key/value pairs to be utf-8 encoded #82

Closed buxingzhe closed 2 years ago

buxingzhe commented 3 years ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] Our application shows the log like below due to the URL path contain non utf-8 encoded characters. "go2sky-gRPC2020/12/28 18:49:46 send segment error rpc error: code = Internal desc = grpc: error while marshaling: proto: field "KeyStringValuePair.Value" contains invalid UTF-8" Describe the solution you'd like A clear and concise description of what you want to happen. The sdk should always encode the key/value string for tags with utf-8.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

wu-sheng commented 3 years ago

the URL path contain non utf-8 encoded characters.

Why is that? If it is URL encoded, it should have UTF-8 encoded already.

buxingzhe commented 3 years ago

As the log doesn't output the real string value, so I'm not sure why this happened. I think it's probably related to the setting of "span.Tag(go2sky.TagURL, req.URL.String()). As the URL may have query parameters that contain characters like Chinese.

wu-sheng commented 3 years ago

URL includes parameters? Which framework having this feature? URL should be encoded even for Chinese.

buxingzhe commented 3 years ago

Here is a URL that with query parameters. Query parameters are the key value pairs after '?'. https://www.baidu.com/s?wd=中文&rsv_spt=1&rsv_iqid=0xee4f0d5400029778

wu-sheng commented 3 years ago

OK, we should use uri#encoding before put it into the tags.

arugal commented 3 years ago

image Hi @buxingzhe Can you give me more information? I didn't come across proto: field "KeyStringValuePair.Value" contains invalid UTF-8 :)

buxingzhe commented 3 years ago

I just found such errors in our log, but I don't figure out how it happens. Here are some links related to this issue, hope to help you. https://github.com/golang/protobuf/issues/1228 https://stackoverflow.com/questions/15139449/google-protocol-buffer-error-encountered-string-containing-invalid-utf-8-data