Iss463 quarantine full ds

[mah-databricks]

What changes are proposed in this pull request?

Rework Piper to quarantine only the partitions which fail. This required a small rewrite to the Iterator code which calls the external CLI tooling. Also needed to backfill exceptions to make sure the application gives the appearance of still functioning the same way.

How is this patch tested?

• [x ] Unit tests
• [ ] Integration tests
• [ x] Manual tests

(Details)

[codecov[bot]]

Codecov Report

Merging #469 (ea36ea2) into master (a7db6b6) will decrease coverage by 0.00%. The diff coverage is 95.23%.

@@            Coverage Diff             @@
##           master     #469      +/-   ##
==========================================
- Coverage   93.66%   93.66%   -0.01%     
==========================================
  Files          95       95              
  Lines        4849     4875      +26     
  Branches      463      457       -6     
==========================================
+ Hits         4542     4566      +24     
- Misses        307      309       +2     

Impacted Files	Coverage Δ
...rojectglow/transformers/pipe/PipeTransformer.scala	100.00% <ø> (ø)
...scala/io/projectglow/transformers/pipe/Piper.scala	93.51% <94.44%> (-0.60%)	:arrow_down:
...jectglow/transformers/pipe/CSVInputFormatter.scala	100.00% <100.00%> (ø)
...low/transformers/pipe/UTF8TextInputFormatter.scala	100.00% <100.00%> (ø)
...n/scala/io/projectglow/vcf/VCFInputFormatter.scala	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a7db6b6...ea36ea2. Read the comment docs.

[williambrandler]

thanks @mah-databricks, let me merge in, rebuild the project and then perform an integration test on this new quarantine functionality