0.10.1 and 0.10.2 have changed import method of array_choice from 0.10.0

MrPowers / quinn

pyspark methods to enhance developer productivity 📣 👯 🎉

https://mrpowers.github.io/quinn/

Apache License 2.0

597 stars 93 forks source link

0.10.1 and 0.10.2 have changed import method of array_choice from 0.10.0 #197

Closed AmberGibney closed 4 months ago

AmberGibney commented 4 months ago

In 0.10.0 you could, as per many other functions, call quinn.array_choice(), now as it is not in the init.py, it has to be imported from quinn.functions.

Is this intentional behaviour? It seems like a pretty large change from 0.10.0 to 0.10.1.

SemyonSinchenko commented 4 months ago

@MrPowers I 100% agree. As I mentioned already, if your goal is to make quinn works with SparkConnect, I suggest to do it in a new major release.

MrPowers commented 4 months ago

@AmberGibney - looks like we forgot to add array_choice in this commit: https://github.com/MrPowers/quinn/commit/95dd2b5caecb8f35226a0e42ef58ffa26a7ef180

I will push up a fix. Thanks for reporting this.

This one is unrelated to Spark Connect @SemyonSinchenko

MrPowers commented 4 months ago

Turns out that this one slipped through the cracks because the array_choice test was commented out. I commented that out cause I wasn't able to make the test deterministic on localhost/CI.

MrPowers commented 4 months ago

@AmberGibney - this should be fixed with quinn v0.10.3