lancedb / lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
https://lancedb.github.io/lance/
Apache License 2.0
3.97k stars 224 forks source link

feat: add `commit_batch` API #3142

Closed wjones127 closed 8 hours ago

wjones127 commented 2 days ago

This allows users to commit a sequence of transactions as one new version. Right now, this only supports Append transactions, which are trivial to merge.

Closes #3097

codecov-commenter commented 12 hours ago

Codecov Report

Attention: Patch coverage is 87.30159% with 16 lines in your changes missing coverage. Please review.

Project coverage is 77.91%. Comparing base (8069936) to head (76e974d). Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
rust/lance/src/dataset/write/commit.rs 87.30% 6 Missing and 10 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #3142 +/- ## ========================================== + Coverage 77.89% 77.91% +0.02% ========================================== Files 242 242 Lines 82183 82296 +113 Branches 82183 82296 +113 ========================================== + Hits 64020 64125 +105 + Misses 14973 14965 -8 - Partials 3190 3206 +16 ``` | [Flag](https://app.codecov.io/gh/lancedb/lance/pull/3142/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/lancedb/lance/pull/3142/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb) | `77.91% <87.30%> (+0.02%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


🚨 Try these New Features: