GoogleCloudPlatform / dlp-dataflow-deidentification

Multi Cloud Data Tokenization Solution By Using Dataflow and Cloud DLP
Apache License 2.0
89 stars 53 forks source link

Added automated test for ORC inspection #175

Closed chitara-01 closed 10 months ago

chitara-01 commented 10 months ago

Summary (Short summary of what is being done) :

Bug fix, automated tests and unit tests for ORC

Description (Describe in detail the fix made) :

  1. Fixing the NullPointer issue with ORC file processing.
  2. Added CI verification tests and unit testing for ORC inspection and de-identification.

    Bug ID (if any) :

  3. b/307245825
  4. b/301563517

    Public Documentation (if any) :


    TESTED (Test Cases with scenario and description - must have 1 positive and 1 negative scenario) :

  5. Used sample data provided by GSD team to verify the pipeline works as expected for fields with null values.
  6. Please check GitHub PR checks.
codecov[bot] commented 10 months ago

Codecov Report

Merging #175 (92eafc5) into master (b96a509) will increase coverage by 1.06%. The diff coverage is 0.00%.

:exclamation: Current head 92eafc5 differs from pull request most recent head cef918d. Consider uploading reports for the commit cef918d to get more accurate results

@@             Coverage Diff              @@
##             master     #175      +/-   ##
============================================
+ Coverage     12.37%   13.43%   +1.06%     
- Complexity       63       67       +4     
============================================
  Files            53       53              
  Lines          2521     2515       -6     
  Branches        209      211       +2     
============================================
+ Hits            312      338      +26     
+ Misses         2190     2157      -33     
- Partials         19       20       +1     
Files Coverage Δ
...n/DLPTextToBigQueryStreamingV2PipelineOptions.java 0.00% <ø> (ø)
...arm/tokenization/DLPTextToBigQueryStreamingV2.java 0.00% <0.00%> (ø)
...m/google/swarm/tokenization/orc/ORCReaderDoFn.java 35.00% <0.00%> (-1.37%) :arrow_down:
...m/google/swarm/tokenization/orc/ORCWriterDoFn.java 0.00% <0.00%> (ø)

... and 2 files with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more