cdrs that are lists are incorrectly serialised

[ahjones]

Example:

(json:encode-json-to-string (json:decode-json-from-string "{\"items\": [1,2,3]}"))
"[[\"items\",1,2,3]]"

I would have expected that the output would have matched the input, rather than a vector of a vector.

The issue though isn't that the JSON can't be round-tripped:

(json:encode-json-to-string '((:items . (1 2 3))))
"[[\"items\",1,2,3]]"

[sirherrbatka]

(1 . (2 3)) is the same as (1 2 3). Nothing wrong here.

[ahjones]

The issue could be clearer (coming back to this after ~1 year I realise it's hard to read). The problem is that I expected that (json:encode-json-to-string (json:decode-json-from-string "{\"items\": [1,2,3]}")) would return a JSON object with a key "items", but instead the return is a vector that contains a vector the first element of which is a string "items".

[diegogub]

I'm having the same issue with objects with every key as array. I solved it using hashtable, but it's not nice..

[rpgoldman]

Part of the issue is this:

CL-USER> (json:decode-json-from-string "{\"items\": [1,2,3]}")
((:ITEMS 1 2 3))

According to the documentation, this function should

Read a JSON Value from json-string and return the corresponding Lisp value. [Italics mine]

But it looks to me like this is trying to return a list of JSON values.

Then the (:items 1 2 3) is not recognized as an association list, so it gets treated as an array. That sort of makes sense -- there isn't a way to detect an association list, which is defined by the spec as:

association list n. a list of conses representing an association of keys with values, where the car of each cons is the key and the cdr is the value associated with that key.

That's not a detectable property -- try writing association-list-p and you will see what I mean. The CL-JSON docs should not promise to translate association lists to/from JSON objects, because detecting association lists is impossible.

To get what you want, you'll have to use with-object and encode-object-member to make the encoding work, AFAICT. It would be a useful addition to have an encode-alist-as-object function to encapsulate this (MR anyone?), but I don't believe there is one.

We should close this issue because this is not a bug -- it's just us expecting CL-JSON to be able to do more than it can do.

[erikronstrom]

Conversions between lisp objects and json objects are inherently ambigous as [], null and false all correspond to nil.

with-explicit-encoder is a way to circumvent these problems when encoding. Since a corresponding decoder has been missing, I made a small extension at some point, see below. Feel free to use or extend it further!

(I could have made a PR of it but since no PRs have been merged since 2014 I didn't feel enough motivation)

(defun json::json-boolean-to-explicit-lisp (token)
  (cond
   ((string= token "true")  '(:true))
   ((string= token "false") '(:false))
   ((string= token "null")  '(:null))))

(defmacro json::with-explicit-decoder (&body body)
  `(json:bind-custom-vars
    (:end-of-array  (lambda () (list* :array (cdr json::*accumulator*)))
     :end-of-object (lambda () (list* :object (cdr json::*accumulator*)))
     :boolean #'json::json-boolean-to-explicit-lisp)
     ,@body))

(defmacro json::with-explicit-semantics (&body body)
  `(json:with-explicit-encoder
     (json::with-explicit-decoder
       ,@body)))

(json::with-explicit-semantics
  (json:encode-json-to-string
   (json:decode-json-from-string "{\"array\": [], \"object\": {}, \"null\": null, \"false\": false, \"true\": true}")))

=> "{\"array\":[],\"object\":{},\"null\":null,\"false\":false,\"true\":true}"