sjrusso8 / spark-connect-rs

Apache Spark Connect Client for Rust

https://docs.rs/spark-connect-rs

Apache License 2.0

76 stars 13 forks source link

feat: client improvements & flaky tests #23

Closed sjrusso8 closed 5 months ago

sjrusso8 commented 5 months ago

Description

feat: client improvements & flaky tests

Update client mod to be more reusable
Leverage environment variables for SPARK_REMOTE and USER
Move SparkSessionBuilder to session module
Change mutex to rwlock when creating the client object
Fix flaky tests
Remove arrow as a re-export