aodn / python-aodntools

Repository for templates and code relating to generating standard NetCDF files for the Australia Ocean Data Network
GNU Lesser General Public License v3.0
10 stars 3 forks source link

Validation, completeness and consistency checks before writing netcdf #30

Closed mhidas closed 5 years ago

mhidas commented 5 years ago

Fixes #14, implementing the apporach described there (https://github.com/aodn/aodn-netcdf-tools/issues/14#issuecomment-438877417)

codecov[bot] commented 5 years ago

Codecov Report

Merging #30 into master will increase coverage by 15.34%. The diff coverage is 100%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master      #30       +/-   ##
===========================================
+ Coverage   76.47%   91.81%   +15.34%     
===========================================
  Files           2        2               
  Lines         204      171       -33     
  Branches       41       29       -12     
===========================================
+ Hits          156      157        +1     
+ Misses         40       11       -29     
+ Partials        8        3        -5
Impacted Files Coverage Δ
ncwriter/template.py 90.27% <100%> (+15.54%) :arrow_up:
ncwriter/schema.py 100% <100%> (ø) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 9ebfadb...b888ef8. Read the comment docs.

mhidas commented 5 years ago

I've added an example script to better demonstrate how this works.

mhidas commented 5 years ago

@lbesnard @bpasquer @ggalibert @ocehugo @lwgordonimos I'm going to merge this so we can build it and it's easier to try it out. Feel free to review and comment anyway. There's obviously more work to do, but it should start to be useable in a basic way.

mhidas commented 5 years ago

You can just pip install the wheel from Jenkins: https://jenkins.aodn.org.au/view/ncwriter/job/ncwriter_build/3/artifact/dist/ncwriter-0.0.6-py2-none-any.whl

ghost commented 5 years ago

@mhidas the following mechanism would be more appropriate for checking things like the numpy types:

https://lat.sk/2017/03/custom-json-schema-type-validator-format-python/ https://python-jsonschema.readthedocs.io/en/latest/validate/#jsonschema.FormatChecker