search

awslabs / python-deequ

Python API for Deequ
Apache License 2.0
713 stars 134 forks source link

chore: Setting the default spark_version value from pyspark.__version__ #175

Open aagumin opened 10 months ago

aagumin commented 10 months ago

*Issue #170

Description of changes: The default value for the SPARK_VERSION variable will be taken from pyspark.__version__. In case of problems, the user also sets the environment variable himself

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

chenliu0831 commented 10 months ago

Test is failing - @Argo3V5A would you take a look?