search

awslabs / python-deequ

Python API for Deequ
Apache License 2.0
713 stars 134 forks source link

Fix Spark version detection for automatic Deequ maven coordinate definition #114

Closed lecardozo closed 1 year ago

lecardozo commented 1 year ago

Issue #, if available: https://github.com/awslabs/python-deequ/issues/111

Description of changes: As discussed in #111, we are reverting the Spark version detection method for Deequ's maven coordinate definition, as the current solution is broken.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

chenliu0831 commented 1 year ago

@lecardozo thanks for the contribution - but I noticed the CI was actually broken, can you take a look? Do we need to add the env var back?

lecardozo commented 1 year ago

Yeah, I just noticed a small typo on the tests! I'll open a new PR with a quick fix 😄