Open bruffridge opened 3 years ago
Resolved with the deployment of the PeTaL Labeler pipeline repo: https://github.com/nasa-petal/petal-labeler-data-pipeline
However, the image shows the inclusion of a JSON DIFF report. I've mentioned this in the README file within the above repository under future works. @bruffridge I'd still say this could be closed and just have a separate task for adding in a JSON DIFF stage to the pipeline (Or just edit #94 ).
Use Great Expectations or a custom script that takes the golden.json and generates a data quality report showing:
Missing data, duplicate data, or data formatting issues. See Herb's google doc for an example report.