brettviren / moo

ruminants on module oriented programming
GNU General Public License v3.0
4 stars 4 forks source link

Update for jsonschema v3.2.0 #46

Closed plasorak closed 10 months ago

plasorak commented 10 months ago

The latest moo version doesn't work with jsonschema==3.2.0, which the DAQ uses. We now have go in the sw stack, and it'd be very interesting to use gojsonnet rather than what we have now (>4 fold speed increase in python) for {fd,nd}daqconf.

I've commented out the offending lines. I'm not actually sure what this should be.

I've also changed the regexes and removed the escaping of . and - chars.

I'm not sure if I have broken anything.

plasorak commented 10 months ago

For reference here is the output of time with and without the latest moo:

(dbt) np04-srv-019:~/NFD23-10-05/runarea > time fddaqconf_gen -c config.json  -m dro.json -f conf_dir # after setting up moo with the current develop
<snip>
real    0m5.389s
user    0m6.383s
sys     0m0.936s
(dbt) np04-srv-019:~/NFD23-10-05/runarea > time fddaqconf_gen -c config.json  -m dro.json -f conf_dir # after resetting moo==0.6.3
<snip>
real    0m23.650s
user    0m22.140s
sys     0m0.744s
plasorak commented 10 months ago

I've also changed the regexes and removed the escaping of . and - chars.

The reason for this is the warning message that shows up in our integration tests:

<string Host>:4
  <string Host>:4: DeprecationWarning: invalid escape sequence '\-'

<string ipv4>:4
  <string ipv4>:4: DeprecationWarning: invalid escape sequence '\.'

-- Docs: https://docs.pytest.org/en/stable/warnings.html

I've assumed that the same would apply for hiername and dotpath (although we don't use them in DUNE-DAQ and they don't show up).

brettviren commented 10 months ago

Okay, some initial confusion but I think I get the situation now.

The format_checker (eg jsonschema.Draft7Validator.FORMAT_CHECKER) is not deprecated but rather is missing in the old version of jsonschema (3.2.0) that DAQ apparently uses. I pick up 4.19.1 on a fresh install today.

I've just pushed fixes that allow moo to work with the old and new jsonschema. But, beware when the old 3.2.0 is used then any format=... constraint will simply be ignored by jsonschema. This may allow bad strings to be accepted as valid. A warning is printed to stderr on the initial load of the moo.jsonschema wrapper module when format_checker is not available.

I've added more tests to cover these issues, though existing ones should have failed. As is of HEAD now, tests work for old and new version of jsonschema.

The regex changes seem fine.

I'll close this ticket but feel free to reopen it if I miss some point.

plasorak commented 10 months ago

The format_checker (eg jsonschema.Draft7Validator.FORMAT_CHECKER) is not deprecated but rather is missing in the old version of jsonschema (3.2.0) that DAQ apparently uses. I pick up 4.19.1 on a fresh install today.

Huh, that's not good, maybe we should update it in the DAQ.

Thanks for quick turnaround!