bug: `NodeSets.toJSON()` produces invalid JSON

[joni-herttuainen]

Long story short, I was looking into updating circuit node sets with simulation ones. I noticed that there is a bug in NodeSets.toJSON() causing it to produce invalid JSON in a cornercase.

Short explanation

The case in hand: if one of the queries is empty, such as node_id in the repro below, the output JSON is invalid in two ways. First of all, the output is not valid JSON (trailing comma raises an exception in the parser). Secondly, the output is invalid in the sense that

{"node_id": [], "mtype": "L6_Y"} 

and

{"mtype": "L6_Y"} 

would return different results.

Repro

import libsonata
import json

node_sets = {
    "Empty_L6_Y": {
        "node_id": [],
        "mtype": "L6_Y"
    }
}

ns_str = json.dumps(node_sets)
print(f"Original:\n{ns_str}\n")

ns = libsonata.NodeSets(ns_str)
converted = ns.toJSON()
print(f"Converted by libsonata:\n{converted}\n")

# raises
json.loads(converted)

output:

Original:
{"Empty_L6_Y": {"node_id": [], "mtype": "L6_Y"}}

Converted by libsonata:
{
  "Empty_L6_Y": {"mtype": ["L6_Y"],  }
}

Traceback (most recent call last):
  File "/gpfs/bbp.cscs.ch/data/scratch/proj30/home/herttuai/NSETM_2235/test.py", line 20, in <module>
    json.loads(converted)
  File "/gpfs/bbp.cscs.ch/ssd/apps/bsd/2023-02-23/stage_externals/install_gcc-12.2.0-skylake/python-3.10.8-rvn6l5/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/gpfs/bbp.cscs.ch/ssd/apps/bsd/2023-02-23/stage_externals/install_gcc-12.2.0-skylake/python-3.10.8-rvn6l5/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/gpfs/bbp.cscs.ch/ssd/apps/bsd/2023-02-23/stage_externals/install_gcc-12.2.0-skylake/python-3.10.8-rvn6l5/lib/python3.10/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 2 column 38 (char 39)

[joni-herttuainen]

Furthermore, if the contents of the passed node sets are empty, NodeSets.toJSON() outputs '\n}' instead of '{}' or '{\n}':

>>> import libsonata
>>> libsonata.NodeSets('{}').toJSON()
'\n}'

[mgeplf]

cool, thanks for the report.

[mgeplf]

should be fixed by https://github.com/BlueBrain/libsonata/pull/289

[mgeplf]

fixed by #289