sjrusso8 / spark-connect-rs

Apache Spark Connect Client for Rust

https://docs.rs/spark-connect-rs

Apache License 2.0

52 stars 11 forks source link

Implement `DataFrameWriterV2` #16

Closed sjrusso8 closed 1 month ago

sjrusso8 commented 2 months ago

Description

The initial DataFrameWriter is created but there is also another way to write data via DataFrameWriterV2. This method has a slightly different implementation and leverages a different proto command message.

I think the methods should mirror the ones found on the Spark API guide, and a new method should be added onto the DataFrame for writeTo.

hntd187 commented 2 months ago

This looks fairly straightforward, I can pick it up.

sjrusso8 commented 2 months ago

Sounds good! Assigned to you. Thanks!

abrassel commented 1 month ago

Hi @hntd187 , are you currently working on this issue?

sjrusso8 commented 1 month ago

Closed #30