Unicode error token reporting

[quackenbush]

When encountering errors at multi-byte/unicode tokens, the ojg parser reports an error message with the wrong character.

$ echo '→' | oj
*-*-* unexpected character 'â' at 1:1

I'd like to either display the correct unicode character, or omit it:

*-*-* unexpected character '→' at 1:1

Note: this same bug shows up in the "encoding/json" library and in Firefox JS, but is correct in Chrome JS.

Here is Chrome's error message:

> JSON.parse('→')

VM83:1 Uncaught SyntaxError: Unexpected token → in JSON at position 0
    at JSON.parse (<anonymous>)
    at <anonymous>:1:6

[ohler55]

I'll take a look. If the unicode can be preserved I'll do that but failing that maybe leave it off.

[ohler55]

Fixed in branch bug/unicode-in-error. Please give it a try and then I'll release. It was easier to fix than expected.

[quackenbush]

Thanks Peter for the quick fix.

Confirmed:

*-*-* unexpected character '→' at 1:1

[quackenbush]

Looks like the "escape" code path needs the same fix:

$ echo '"\→"' | ./main
*-*-* invalid JSON escape character '\â' at 1:3

[ohler55]

Considered that but chose wrong. On it.

[ohler55]

Ok, pushed that change too.

[quackenbush]

Yep, looks good.

[ohler55]

Released