Configuration Packages are not JSON serializable

drodarie commented 3 months ago

from bsb import Scaffold, parse_configuration_content
conf = {
  "packages": ["cerebellum==0.0.1"],
  "storage": {"engine": "hdf5"},
  "network": {"x": 200, "y": 200, "z": 200},
  "partitions": {},
  "cell_types": {},
  "placement": {},
  "connectivity": {}
}
cfg = parse_configuration_content(conf)
network = Scaffold(cfg)

raises the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/toromis/Workspace/dbbs/bsb/bsb-core/bsb/core.py", line 140, in __init__
    self._bootstrap(config, storage, clear=clear)
  File "/home/toromis/Workspace/dbbs/bsb/bsb-core/bsb/core.py", line 192, in _bootstrap
    self.storage = storage
  File "/home/toromis/Workspace/dbbs/bsb/bsb-core/bsb/core.py", line 215, in storage
    storage.init(self)
  File "/home/toromis/Workspace/dbbs/bsb/bsb-core/bsb/storage/__init__.py", line 340, in init
    self.store_active_config(scaffold.configuration)
  File "/home/toromis/Workspace/dbbs/bsb/bsb-core/bsb/storage/__init__.py", line 261, in store_active_config
    return self._engine.files.store_active_config(config)
  File "/home/toromis/Workspace/dbbs/bsb/bsb-hdf5/bsb_hdf5/file_store.py", line 109, in store_active_config
    active_id = self.store(json.dumps(config.__tree__()), meta)
  File "/usr/lib/python3.10/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.10/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.10/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Requirement is not JSON serializable

This happens because Requirement class does not have a __tree__ or __inv__ function and is therefore not properly parsed to a tree.

drodarie commented 3 months ago

Should we create a Node class for Requirements? or should tree_of returns str(value) if the __inv__ and __tree__ are not defined.

Helveg commented 3 months ago

I think the best way is for store_active_config to use bsb-json instead of just json.dumps. If the issue persists we can solve it centrally from bsb-json with a custom JSON encoder. Falling back to str(value) seems detrimental because it would silence the location of similar issues in the future.

The PackageRequirement type handler does have an __inv__ method, so maybe track down why it's not being invoked here, fix it, and add tests. This is one of those things that I feel like I've "fixed" 2 or 3 times already but fixing 1 things breaks another, and without tests we won't be able to pin down both issues at the same time :D

dbbs-lab / bsb-core

Configuration Packages are not JSON serializable #842