lancedb / lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
https://lancedb.github.io/lance/
Apache License 2.0
3.97k stars 224 forks source link

chore: fix miniblock selection logic error #3107

Closed broccoliSpicy closed 1 week ago

broccoliSpicy commented 1 week ago

This PR tries to fix a logic error during selecting miniblock encoder.

github-actions[bot] commented 1 week ago

ACTION NEEDED Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

broccoliSpicy commented 1 week ago

This is based on TPC-H testing right? Do you need <= 256? Or is < 256 ok? I thought one of the columns had a max length of 256?

in the TPC-H schema, the largest string length it has is 199 in

CREATE TABLE partsupp
(
    ps_partkey     BIGINT not null,
    ps_suppkey     BIGINT not null,
    ps_availqty    BIGINT not null,
    ps_supplycost  DOUBLE PRECISION  not null,
    ps_comment     VARCHAR(199) not null
);
codecov-commenter commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 77.11%. Comparing base (3f2faf2) to head (37fe3bf).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #3107 +/- ## ========================================== - Coverage 77.13% 77.11% -0.02% ========================================== Files 240 240 Lines 80652 80643 -9 Branches 80652 80643 -9 ========================================== - Hits 62208 62188 -20 + Misses 15275 15274 -1 - Partials 3169 3181 +12 ``` | [Flag](https://app.codecov.io/gh/lancedb/lance/pull/3107/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/lancedb/lance/pull/3107/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb) | `77.11% <100.00%> (-0.02%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.