search

ioos / compliance-checker

Python tool to check your datasets against compliance standards
http://ioos.github.io/compliance-checker/
Apache License 2.0
108 stars 58 forks source link

Dataset from zarr #1071

Closed ocefpaf closed 4 months ago

ocefpaf commented 4 months ago

Closes #884 and #672.

@benjwadams I had to squash the original 17 commits to make this manageable but I believe I got everything in.

PS: This implements checker for nczarr! Not pure zarr datasets. The latter would be easier to implement when we move away from netCDF files as the base for compliance checking.

codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 60.60606% with 26 lines in your changes are missing coverage. Please review.

Project coverage is 81.75%. Comparing base (6a006e4) to head (7704919). Report is 1 commits behind head on develop.

Files Patch % Lines
compliance_checker/protocols/zarr.py 71.79% 6 Missing and 5 partials :warning:
compliance_checker/protocols/netcdf.py 30.76% 4 Missing and 5 partials :warning:
compliance_checker/protocols/opendap.py 50.00% 3 Missing and 1 partial :warning:
compliance_checker/suite.py 66.66% 1 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #1071 +/- ## =========================================== - Coverage 81.92% 81.75% -0.17% =========================================== Files 24 25 +1 Lines 5172 5224 +52 Branches 1237 1253 +16 =========================================== + Hits 4237 4271 +34 - Misses 634 644 +10 - Partials 301 309 +8 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

ocefpaf commented 4 months ago

@benjwadams only 1 test is failing, TestCLI.test_nczarr_pass_through[local_file], with:

                                     Errors                                     
--------------------------------------------------------------------------------
§2.1 Filename
* Dataset path file:///home/filipe/Dropbox/pymodules/01-forks/IOOS/compliance-checker/compliance_checker/tests/data/trajectory.zarr#mode=nczarr,file must end with ".nc"

That is something we need to skip b/c zarr files are, in a way, against that rule.

BTW, I could not make the skip option work. Not sure that is going on...

I skipped two tests: check_filename b/c .zarr won't pass a name test for .nc, and check_coordinate_variables_strict_monotonicity b/c I believe that openSourcerer9000 original dataset has a real issue that we need to fix to run the tests.

With that said, this PR works 🎉 and is read for a review!