Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.
We should update Koheesio to be able to rely on this and update our exceptions so that v2.2.0 (or greater) is advised when we're on Spark Connect and using SE Transformations. Also - we should test our koheesio SE implementation (in general) against the new version of SE.
SparkExpectations 2.2.0 just released bringing in support for Spark Connect https://github.com/Nike-Inc/spark-expectations/releases/tag/v2.2.0
We should update Koheesio to be able to rely on this and update our exceptions so that v2.2.0 (or greater) is advised when we're on Spark Connect and using SE Transformations. Also - we should test our koheesio SE implementation (in general) against the new version of SE.