BUG: Bring IdentifiableScan into precise equivalence with other maskers, e.g., Detection.RedactionToken is now in alignment.
NEW: Provide hybrid capability to run high-performance detections in IdentifiableScan and fall back to other masker as required.
This change extends our unit testing to ensure that the high performance Rust library is exactly equivalent in behavior to the other libraries. I noticed a discrepancy in some ad hoc testing in redaction token production. Testing also closed a few other holes.
Opportunistically, to enable full testing I updated the high perf scanner to create a back-up masker for unimplemented rules. This allows the high performance masker to be initialized with arbitrary patterns. If it can assume responsibilities for detection, it does so, otherwise it uses a fallback masker.
IdentifiableScaninto precise equivalence with other maskers, e.g.,Detection.RedactionTokenis now in alignment.IdentifiableScanand fall back to other masker as required.This change extends our unit testing to ensure that the high performance Rust library is exactly equivalent in behavior to the other libraries. I noticed a discrepancy in some ad hoc testing in redaction token production. Testing also closed a few other holes.
Opportunistically, to enable full testing I updated the high perf scanner to create a back-up masker for unimplemented rules. This allows the high performance masker to be initialized with arbitrary patterns. If it can assume responsibilities for detection, it does so, otherwise it uses a fallback masker.