Create similar bindings as with Rust but available in server side js (node, deno, bun, ...). The sdk should closely resemble the rust one, and only deviate when either necessary due to napi limitations, or when it is unidiomatic in JS.
napi.rs seems to be good crate to leverage and is relatively easy to use.
Early Experiment
The branch feat/napi contains a super quick pass at creating the bindings. The experiment only covers these areas
Create a remote SparkSession
Create a dataframe with .sql
Modify the dataframe with select, and filter
Perform "action" with count()
Perform “action” with show()
There is a lot of use of clone() and some not great implementations to create a new empty dataframe to satisfy the napi requirements. The polars js interop is a good example of how the bindings might function.
Description
Create similar bindings as with Rust but available in server side js (node, deno, bun, ...). The sdk should closely resemble the rust one, and only deviate when either necessary due to napi limitations, or when it is unidiomatic in JS.
napi.rs seems to be good crate to leverage and is relatively easy to use.
Early Experiment
The branch feat/napi contains a super quick pass at creating the bindings. The experiment only covers these areas
.sqlselect, andfiltercount()show()There is a lot of use of
clone()and some not great implementations to create a new empty dataframe to satisfy the napi requirements. The polars js interop is a good example of how the bindings might function.