Closed MrPowers closed 2 months ago
Whoops, this is actually working, I forgot to start the Spark Connect session with the command: ./sbin/start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.1
Are you OK if I add these setup instructions to the README?
So I just ran through the steps on a fresh ubuntu image, and here is what I had to do.
rustup
build-essential
& protoc
with apt install -y build-essential protobuf-compiler
cmake
cargo new spark-connect-starter && cd spark-connect-starter
to create a new project and go to the directoryCargo.toml
filespark-connect-rs = "0.0.1-beta.3"
tokio = "1.37.0"
main.rs
and run cargo run
Ensure spark connect is up and running.
I'll update the top level readme with the example from the gist. The current one leverages a file path that might not exist for all users depending on how they are running their spark cluster.
I see you also updated the spark-connect-go
readme. I'll mirror that style
Commands I ran (I am a total n00b):
rustup
cargo new spark-connect-gist
Cargo.toml
to include spark connect rs andtokio
dependencies:arch -arm64 brew install cmake
brew install protobuf
main.rs
: https://gist.github.com/sjrusso8/2b4e43af462367a15f91db5a33627449cargo run
Here is the error message: