Closed ghukill closed 6 years ago
Consider, also, whether these "valves" should be at the Record Validation Scenario (RVS) level. Could have toggles for each RVS, where toggling "on" would mean to include. Default could be to all, or none. Noting here, but likely better to consider once in place.
Mostly done. Closing this issue, preparing to merge with dev
. Below is a graph showing a scenario of how this might help, where the Transform --> Merge Job.
Reopening - work on GUI for this addition still ongoing.
Todo
Adding counts to edges and tables based on record validity
Done.
Related to #119
Allowing the user to select all, invalid, or valid records from an input job, opens up a handful of useful workflows.
If a Job contains a large number of records, but a small number fail validation, it would be beneficial to allow a TransformationJob to operate over only the fail records, and then use a MergeJob to recombine. In some situations, this may be required, where a Transformation would alter valid records in an undesirable way.
The following two scenarios outline how the use of all / invalid / valid "valves" could be used:
First step will be to allow these valves on input jobs, then look into how the UI might help users merge the disparate pieces back together (which could, of course, be done manually).