search

rudolph-miller / jonathan

JSON encoder and decoder.
http://rudolph-miller.github.io/jonathan/overview.html
164 stars 24 forks source link

"Array index 43 out of bounds for…" error when parsing json with escaped unicode symbols #52

Closed svetlyak40wt closed 5 years ago

svetlyak40wt commented 6 years ago

Minimal example, which does not work:

(jonathan:parse "\"\\uD83D\\uDE3E\\uD83D\\uDD2A\"")

But separate characters work:

CL-USER> (jonathan:parse "\"\\uD83D\\uDE3E\"")
"😾"
CL-USER> (jonathan:parse "\"\\uD83D\\uDD2A\"")
"🔪"
CL-USER>
svetlyak40wt commented 6 years ago

Tested on SBCL and it works.

OS:   Darwin 15.6.0
Lisp: Clozure Common Lisp Version 1.11.5/v1.11.5  (DarwinX8664)
ASDF: 3.3.1.1
QL:  quicklisp 2018-03-28

Tested on CCL and it fails.

OS:   Darwin 15.6.0
Lisp: Clozure Common Lisp Version 1.11.5/v1.11.5  (DarwinX8664)
ASDF: 3.3.1.1
QL:  quicklisp 2018-03-28
svetlyak40wt commented 6 years ago

Rechecked the output under SBCL, and it is wrong. SBCL does not work but in other way.

I've added a test:

(is (parse "\"\\uD83D\\uDE3E\\uD83D\\uDD2A\"")
        (concatenate 'string
                     (parse "\"\\uD83D\\uDE3E\"")
                     (parse "\"\\uD83D\\uDD2A\""))
        "surrogate pairs should be parsed equally when they follow together and separate from each other."))

And under SBCL it fails by producing this error:

"😾u" is expected to be "😾🔪"
svetlyak40wt commented 6 years ago

By the way, cl-json is unable to parse this string either. Not under SBCL nor under CCL.

svetlyak40wt commented 5 years ago

Oh, this issue was fixed in a pull https://github.com/Rudolph-Miller/jonathan/pull/53 one year ago!

Closing. Thanks to https://12forks.com MVP for pointing to it.