This dockerfile can be used to setup and run the tests in the Python Deequ package. This way, we do not need to install any dependencies in our local workspaces. Right now, it only builds against Spark version 3.3. Will be adding other versions in a future PR.
Verified that the docker run output is the same as that of the PR workflow.
Description of changes:
This dockerfile can be used to setup and run the tests in the Python Deequ package. This way, we do not need to install any dependencies in our local workspaces. Right now, it only builds against Spark version 3.3. Will be adding other versions in a future PR.
Verified that the docker run output is the same as that of the PR workflow.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.