ACCESS-NRI / access-nri-intake-catalog

Tools and configuration info used to manage ACCESS-NRI's intake catalogue
https://access-nri-intake-catalog.rtfd.io
Apache License 2.0
8 stars 1 forks source link

Metadata validation provides more complete report #231

Closed marc-white closed 3 weeks ago

marc-white commented 3 weeks ago

Closes #216 .

This PR modifies the behaviour of the function utils.validate_against.schema so that a report of all metadata schema errors is provided, rather than just the first issue found.

The function still raises jsonschema.ValidationError, but the error message is now formatted as follows:

01 | <field name or (missing) > : <reason for failure>
02 | <field name or (missing) > : <reason for failure>
...

For example, the error message for a metadata file that has a bad value for name, and no value at all for experiment_uuid, would be:

01 | name : None is not of type 'string'
02 | (missing) : 'experiment_uuid' is a required property

This behaviour then flows through to the metadata-validate command-line script. The returns of this script have also been refined to:

Tests for the new error handling have also been included. These tests are exhaustive, but not complete (I didn't think it was necessary to provide every single possible permutation).

codecov[bot] commented 3 weeks ago

Codecov Report

Attention: Patch coverage is 70.00000% with 6 lines in your changes missing coverage. Please review.

Project coverage is 96.36%. Comparing base (bbee7f6) to head (0471031). Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/access_nri_intake/cli.py 40.00% 6 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #231 +/- ## ========================================== - Coverage 97.17% 96.36% -0.81% ========================================== Files 9 9 Lines 672 688 +16 ========================================== + Hits 653 663 +10 - Misses 19 25 +6 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

charles-turner-1 commented 3 weeks ago

I've checked out to this branch & run the tests & all the additional tests seem to be failing on my machine - I'd just like to do a tad more prodding to work out why before I approve the changes. Bit strange given they're working on CI...

marc-white commented 3 weeks ago

@charles-turner-1 what was the environment issue you ran into?

charles-turner-1 commented 3 weeks ago

@marc-white just a pytest extension (either watcher or sugar, not sure which - I rebuilt the environment without either) capturing the exception and assuming it was wrapped with an ExceptionInfo, rather than a bare exception - caused the expected errors to get turned into TypeErrors.