With these changes applied, I was able to run all six new examples in the vectorpipe.examples directory. These bugs were likely fallout from the GT 3.1 / Spark 2.4 upgrade and were only exposed with runnable examples in the project.
Testing
Run each of the examples using default arguments, they should now all pass. For example:
With these changes applied, I was able to run all six new examples in the
vectorpipe.examplesdirectory. These bugs were likely fallout from the GT 3.1 / Spark 2.4 upgrade and were only exposed with runnable examples in the project.Testing
Run each of the examples using default arguments, they should now all pass. For example:
For the AugmentedDiff examples you'll need to correctly set AWS_PROFILE and pass an s3 augmented diff location for
--augmented-diff-source.