FRosner / drunken-data-quality

Spark package for checking data quality
Apache License 2.0
222 stars 69 forks source link

Release DDQ 5.0.0 #130

Open FRosner opened 7 years ago

FRosner commented 7 years ago
jordanyaker commented 4 years ago

Any update on when this new build will be released?

FRosner commented 4 years ago

Hi! I am not actively working on the project at the moment so there are no plans to release a new version at the moment. Is there any specific feature you are looking forward to?

jordanyaker commented 4 years ago

@FRosner the only thing that's a blocker to me using this is #152, however, it looks like that's been fixed in master but not in 4.1.1.

FRosner commented 4 years ago

I see. In this case we could release 4.1.2 with only that fix. Would that help? If so, I can do that tomorrow.

jordanyaker commented 4 years ago

That would totally help!

FRosner commented 4 years ago

Hi! I don't have time today as something else took longer than expected. I will put a reminder for Monday, though. Sorry!

FRosner commented 4 years ago

I released 5.0.0 but I am having some issues with Travis. Nevertheless the package has been published. Can you check if it works for you @jordanyaker?

jordanyaker commented 4 years ago

@FRosner sorry for the delay - I've been dealing with migration problems for the last two weeks. I finally got back to this task and was able to check on this.

I installed 5.0.0 Python library without any issues. I couldn't find the 5.0.0 JAR so, I installed 4.1.1. Unfortunately, it looks like this is a problem due to the following error:

  File "/local_disk0/spark-047d81fc-c47d-4e05-a823-339d733333aa/userFiles-2d767fef-09dc-4e1a-807b-d9e62df71d41/addedFile2119423679649655466drunken_data_quality_4_1_1_s_2_11-c7bb5.jar/pyddq/core.py", line 439
    print baos.get_output()
             ^
jordanyaker commented 4 years ago

Scratch what I said earlier. I downloaded the 5.0.0 JAR from the Github releases for the project and uploaded it to my cluster. It worked without issues. Thanks!

FRosner commented 4 years ago

Glad it worked! I'll see when I can find the time to fix the travis build.