Round trip load/dump with comment preservation

[erwanp]

Hello @spyoungtech

I'm looking for a JSON parser with round-trip support for comments to use as a configuration file in an open-source physics code.

JSON5 looks like a great candidate; and the doc suggests you have round-trip comment support already ; but I couldnt get it to work in my case.

Example of use-case :

import json5     # !pip install json-five

jsonfilecontent = """{
"database": {                       /* database key, stores all databases */
    "HITEMP-CO2": {                 /* a database name */
    "path": [
        "PATH/TO/1",
        "PATH/TO/2"]}
  }
}"""

config = json5.loads(jsonfilecontent )
config["NEW"] = {"path":["PATH/TO/3"]}      # EXPECTED : edit as a dict 
with open("config.json", "w") as f:
    json5.dump(JSONCDict, f)                    # EXPECTED : dump and keep comments

With json5 the above does not work yet; I understand it's because #12 is not implemented.

I tried based on the round-trip comment example :

from json5.loader import loads, ModelLoader
from json5.dumper import dump, ModelDumper
from json5.model import BlockComment

model = loads(jsonfilecontent, loader=ModelLoader())           # loaded ok
with open("config.json", "w") as f:
    dump(model, f, dumper=ModelDumper())           # comments preserved

It works fine, comments are preserved. However, how can I insert a full Python dictionary as a new entry in the model ?

[spyoungtech]

Hi @erwanp thanks for filing this issue. Glad to see people have an interest in working with JSON5 comments.

Unfortunately, the current interface for modifying raw models is very much in early phases of development and not very user friendly or well-documented.

What I hope to eventually do is create an abstraction layer that adds high-level methods for altering the JSON model objects in similar ways that you might alter the python-equivalent object. This is the idea of #6

Anyhow, for now, modifying the model (like adding new values to an object) requires modifying the object directly. Forewarning: this is tricky and there's likely some landmines laying around to be stepped on.

The way you might go about this is using the modelize function in the dumper module to generate models from python objects, then modifying your original model using attributes of the new model you created.

As a "minimal" example:

from json5.loader import loads, ModelLoader
from json5.dumper import dumps, ModelDumper, modelize
from json5.model import BlockComment

model = loads('{foo: "bar" /*a comment */}', loader=ModelLoader())

my_dict = {'bacon': 'eggs'}  # we're going to merge this with the existing model JSON object.
my_dict_model = modelize(my_dict)
model.value.key_value_pairs.extend(my_dict_model.key_value_pairs)

print(dumps(model, dumper=ModelDumper()))
# {foo: "bar" /*a comment */,'bacon':'eggs'}

Hope that helps.

If you have thoughts on use-cases or what an ideal interface might look like for you, I'm open to suggestions :-)

[FlorianLudwig]

@spyoungtech your comment helped me, was looking for the same as @erwanp :+1:

One suggestion: make the Nodes friendlier to work with.

For example allow accessing a JSONObject without having to iterate over the key value pairs, see https://github.com/spyoungtech/json-five/pull/32

[spyoungtech]

@FlorianLudwig yes, that would be nice! This, among other changes, is tracked as part of #6

32 looks interesting and am looking forward to reviewing once ready

I also plan to add methods for (hopefully safely!) working with comments. While it is possible to edit models as described above, it does not provide any guarantees that the output is valid JSON5.

[FlorianLudwig]

@spyoungtech thanks for the quick follow up!

I marked #32 as ready - the reason I marked it as draft was I wasn't sure if that was the direction you wanted to go or if you prefer a different approach, like an api on top of the nodes for editing.