lancedb / lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..
https://lancedb.github.io/lance/
Apache License 2.0
3.47k stars 180 forks source link

feat: add experimental limit variables for GCS #2295

Closed wjones127 closed 1 month ago

wjones127 commented 1 month ago

This adds experimental environment variables for configuring how to handle retries and parallelism for uploads. We'll see how this works in production workloads before we decide what the permanent API is.

codecov-commenter commented 1 month ago

Codecov Report

Attention: Patch coverage is 0% with 53 lines in your changes are missing coverage. Please review.

Project coverage is 80.62%. Comparing base (27e6cfc) to head (b7edd60). Report is 2 commits behind head on main.

Files Patch % Lines
rust/lance-io/src/object_store/gcs_wrapper.rs 0.00% 53 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2295 +/- ## ========================================== - Coverage 80.66% 80.62% -0.04% ========================================== Files 191 191 Lines 56109 56142 +33 Branches 56109 56142 +33 ========================================== + Hits 45259 45265 +6 - Misses 8318 8350 +32 + Partials 2532 2527 -5 ``` | [Flag](https://app.codecov.io/gh/lancedb/lance/pull/2295/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/lancedb/lance/pull/2295/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb) | `80.62% <0.00%> (-0.04%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.