search

duckdb / duckdb-rs

Ergonomic bindings to duckdb for Rust
MIT License
428 stars 87 forks source link

Duckdb updates #337

Open Swoorup opened 3 weeks ago

Swoorup commented 3 weeks ago
  1. Strong types for duck db vectors.
  2. Ensure correct size are reserved before writing to duckdb vectors.
  3. Introduce DataChunkHandle for manipulating data chunks, and wrap unsafe functions at bay. There are still few handle like duckdb_appender that is better off being wrap instead of sprinkling unsafe everywhere.
  4. Add union, struct type and nested type without the duplicated code. (It should be simple enough to now add other nested types like Map)
  5. Appender keeps a cached copy of the schema for error handling and insertions.
  6. Fix large amount of record insertion (Respect the duckdb limit)
  7. Remove unsafe code for traits that need not be unsafe. Entire idea for rust is to limit usage of unsafe code when providing as a library.
  8. For unit testing, I also added arrow-convert library which makes it easier to derive arrow arrays directly from rust specific unions/structs than hand writing it.
Swoorup commented 3 weeks ago

Fixes or partially fixes the following:

era127 commented 3 weeks ago

I think the 306 update to the test case could be merged into your changes.

Swoorup commented 2 weeks ago

Union feature is blocked by a suitable release i.e which includes the fix https://github.com/duckdb/duckdb/pull/11326 https://github.com/duckdb/duckdb-rs/pull/336