search

gomarkdown / markdown

markdown parser and HTML renderer for Go
Other
1.36k stars 171 forks source link

markdown.Parse cause garbled characters in some Chinese scenarios #305

Closed haolvh closed 5 months ago

haolvh commented 5 months ago

https://github.com/gomarkdown/markdown/blob/1d6d20845b47fa00debbd727d537128b9f0776d2/parser/parser.go#L296-L298

These two lines of code will cause garbled characters in some Chinese scenarios with "\r\n". For example, if the value of input here is the Chinese string "ha\r\n." after []byte("ha\r\n."), after processing these two lines of code, use string(input ), garbled characters will appear.

As shown in the picture:

image

This case is tested on mac.

kjk commented 5 months ago

Please provide a test case. You can start with https://tools.arslexis.io/goplayground/#txO7hJ-ibeU and use Share button to share your modified example.

haolvh commented 5 months ago

example this case：https://tools.arslexis.io/goplayground/#UJ2_QJHqpiy

kjk commented 5 months ago

Thanks. Even more simplified: https://tools.arslexis.io/goplayground/#846Q6bcLmeX

This is quite odd, I don't see why NormalizeNewline() should break this.