Check for setting keys in the stream.settings which are not available

[timonmerk]

This is a common error when specifying the pipeline. Typos are by now to checked

Solution: nm_settings.yaml? Or write individualized getters and setters.

[toni-neurosc]

So if I understand correctly the problem is that sometimes the user might write an "my_settings.json" file with typos right? How would a YAML file help?

For JSON there's this: https://json-schema.org/understanding-json-schema It has an actively maintained Python module. Perhaps if every feature can have it's own schema for validation, we can run all settings sub-dicts against their own validation schemas at the start of the run and return the relevant error message.

The first thing would be to have a clear idea about how many shapes can a configuration value have. I took a quick look at the default settings and so far have this:

• Dict of "string":bool
  • features
• Dict of "string":int
  • [ "bandpass_filter_settings"]["segment_lengths_ms"]
• Dict of "string":float
  • kalman_filter_settings
• Dict of "string":[list of int]
  • frequency_ranges_hz
• Dict of "string":[list of floats]
  • [fooof][peak_width_limits]
• List of lists of ints: [[5, 80], [5, 30]]
  • [sharpwave_analysis_settings][sharpwave_features]
• Dict of "string":[list of "string"]
  • [sharpwave_analysis_settings][estimator]
• List of lists of "string": [["STN_RIGHT_0", "ECOG_RIGHT_0"]]
  • [coherence][channels]
• Dict of "string":"string":
  • "method": "plv"

[timonmerk]

That's really helpful! I wasn't aware of the jsonschema modules. The motivation being yaml is that it already has with pyyaml quite a nice module where the yaml file could directly be ported into a settings object. This is an example from the pyyaml documentation:

yaml.load(""" ... --- !Monster ... name: Cave spider ... hp: [2,6] # 2d6 ... ac: 16 ... attacks: [BITE, HURT] ... """)

Monster(name='Cave spider', hp=[2, 6], ac=16, attacks=['BITE', 'HURT'])

When making changes within python in the settings it would be possible to access the object attributes and not just a dictionary. But maybe a similar method is also possible with jsonschema.

Also, this might be minor, but yaml also has less brackets and seems to be used within other packages for parametrization more than json (but that's just my impression)

[toni-neurosc]

JSONschema also validates YAML once loaded to a dictionary so it's no problem to choose YAML if the file format is more user-friendly or there's a need to change the settings representation from dictionary to object. However, having a class instance instead of a simple data structure seems to add unnecessary complexity unless there is a concrete need to do the swap. Also, unless the settings class contains a bunch of dictionaries, that's a lot of class writing for each feature and sub-feature...

If JSONschema does not fit the needs there's also Pydantic which could have more/different capabilities, haven't done much research yet https://docs.pydantic.dev/latest/concepts/json_schema/

PS. Loving the YAML example, I have been playing Baldur's Gate 3 for a while today :P

[toni-neurosc]

This was SO closed by #329.