The main datafusion crate (called datafusion-core in the repo) is not yet compatible with wasm. This should eventually be doable, but there are issues to work out regarding dependencies with native build requirements.
The current vegafusion-wasm crate depends on vegafusion-core, which in turn depends on datafusion-common. The main reason to depend on this crate currently is for the ScalarValue support.
I realized that the vegafusion-datafusion-udf, vegafusion-dataframe, datafusion-sql (without the datafusion-conn feature enabled), and datafusion-runtime crates all only depend only on the datafusion crates that are compatible with WASM. The only vegafusion dependency on the core datafusion crate is datafusion-sql with the datafusion-conn feature flag enabled. This means that it should be possible to compile the vegafusion-runtime to WASM and use an alternative WASM-compatible connection implementation.
One obvious candidate for a WASM-compatible connection would be a connection to DuckDB WASM. This would be very similar to how the vegafusion Python API includes support for implementing connections in Python.
In https://github.com/apache/arrow-datafusion/pull/7633 I worked through which
datafusion-*crates are currently compatible withwasm-pack. These are:The main
datafusioncrate (calleddatafusion-corein the repo) is not yet compatible with wasm. This should eventually be doable, but there are issues to work out regarding dependencies with native build requirements.The current
vegafusion-wasmcrate depends onvegafusion-core, which in turn depends ondatafusion-common. The main reason to depend on this crate currently is for theScalarValuesupport.I realized that the
vegafusion-datafusion-udf,vegafusion-dataframe,datafusion-sql(without thedatafusion-connfeature enabled), anddatafusion-runtimecrates all only depend only on the datafusion crates that are compatible with WASM. The only vegafusion dependency on the coredatafusioncrate isdatafusion-sqlwith thedatafusion-connfeature flag enabled. This means that it should be possible to compile thevegafusion-runtimeto WASM and use an alternative WASM-compatible connection implementation.One obvious candidate for a WASM-compatible connection would be a connection to DuckDB WASM. This would be very similar to how the vegafusion Python API includes support for implementing connections in Python.