search

bytedance / sonic

A blazingly fast JSON serializing & deserializing library
Apache License 2.0
6.8k stars 333 forks source link

Parse UTF8 characters failed #578

Closed rts-gordon closed 8 months ago

rts-gordon commented 8 months ago

Hi there,

I use Sonic to parse UTF8 characters but failed, parse other UTF8 characters is OK, can you please take a look at this? thank you.

{"level":"error","ts":1705391424.5935595,"msg":"Unmarshal account failed, error: \"Syntax error at index 222: invalid char\\n\\n\\t\\\"Ŀ\\xfc\\\\\\xc7\\xf8\\xc9\\xcfĿ\\xfc\\\\\\\",\\\"state\\\":\\\"\\x96|\\xbe\\xa9\\xb6\\xbc\\\",\\n\\t................^...............\\n\", message: {\"server_id\":\"live01\",\"country\":\"Japan\",\"city\":\"Ŀ\ufffd\\\ufffd\ufffd\ufffd\ufffdĿ\ufffd\\\",\"state\":\"\ufffd|\ufffd\ufffd\ufffd\ufffd\"}
Syntax error at index 222: invalid char

    "Ŀ�\����Ŀ�\","state":"�|����",
    ................^...............
AsterDY commented 8 months ago

It's obviously that your input is not valid UTF8... Try use STD json lib to compare, I guess you will get same error