Feature: Support an Async/Await in Rust Client with Connection Pool for Kuzu DB

ModPhoenix commented 1 month ago

API

Rust

Description

We propose enhancing the Rust client for Kuzu DB by implementing async/await support and adding a connection pool. These improvements would significantly enhance the developer experience (DX) and align Kuzu DB with modern Rust frameworks and practices.

Proposed Changes

Implement async versions of existing synchronous methods in the Rust client.
Add a connection pool to manage and reuse database connections efficiently.
Update the API to support both synchronous and asynchronous usage.

Rationale

Async/Await Support

Implementing async/await support for the Rust client would bring several benefits:

Improved developer experience by aligning with modern Rust asynchronous patterns
Better integration with popular async Rust frameworks like Axum, Actix, and others
Enhanced flexibility in handling concurrent database operations

Connection Pool

Adding a connection pool would:

Simplify connection management in concurrent scenarios
Provide a more idiomatic way of handling connections in async Rust applications
Offer better control over resource usage in high-concurrency environments

Example Usage (Proposed)

use kuzu_async::{Database, SystemConfig, ConnectionPool};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let db = Database::new(path, SystemConfig::default()).await?;
    let pool = ConnectionPool::new(&db, 10).await?; // Create a pool with 10 connections

    let conn = pool.get().await?;
    conn.query("CREATE NODE TABLE Person(name STRING, age INT64, PRIMARY KEY(name));").await?;
    conn.query("CREATE (:Person {name: 'Alice', age: 25});").await?;
    conn.query("CREATE (:Person {name: 'Bob', age: 30});").await?;

    let mut result = conn.query("MATCH (a:Person) RETURN a.name AS NAME, a.age AS AGE;").await?;
    println!("{}", result.display());

    Ok(())
}

Impact

This enhancement would make Kuzu DB more attractive for Rust developers working with modern async frameworks and patterns. It would improve the overall developer experience by providing a more idiomatic way of working with Kuzu DB in async Rust applications.

Additional Considerations

Backward compatibility with the existing synchronous API
Comprehensive documentation and examples for the new async features
Integration examples with popular Rust web frameworks (e.g., Axum, Actix)

We believe these improvements would greatly benefit the Kuzu DB ecosystem and its users by aligning with modern Rust development practices.

prrao87 commented 1 month ago

As a proposed starting point, there's an async_duckdb crate that does something similar for DuckDB. https://docs.rs/async-duckdb/latest/async_duckdb/

It makes sense to create a separate crate for this, and allow users to choose which API they prefer for their application.

prrao87 commented 1 month ago

One important limitation of embedded databases is related to concurrency. We are likely to have the same limitation as DuckDB when it comes to connection pools, where you can only open connection pools for READ_ONLY processes. See this page for the details (they point to a link on concurrency in the DuckDB docs, and we have a similar page on our docs). https://crates.io/crates/async-duckdb

kuzudb / kuzu