NorthArrowResearch / champ-workbench

http://workbench.northarrowresearch.com/
GNU General Public License v3.0
0 stars 0 forks source link

Retrieve validation issues from FAIL_DATA.json #19

Open volkcj opened 7 years ago

volkcj commented 7 years ago

SFR and crews need an efficient way to review validation errors. I believe we've discussed pulling these into Workbench.

MattReimer commented 7 years ago

@volkcj just a point of clarification. Title implies we'll be harvesting this information from cloudwatch. I think that's probably not a good idea.

We have options:

  1. Get it directly from the S3 validation XML files. The risk here is that old XML files aren't currently cleaned up on new runs so you might be seeing stale data.
  2. We could parse more than just the pass/fail status into a metric schema

I'm leaning toward (2) as the best option here. We even already have a metric schema set up for this:

https://github.com/SouthForkResearch/CHaMP_Metrics/blob/master/xml/TopoVisitValidation.xml

It needs to be filled out and then everything to do with validation can be harvested off the API.

Thoughts?