Closed v0y4g3r closed 3 months ago
Attention: Patch coverage is 82.87037%
with 111 lines
in your changes missing coverage. Please review.
Project coverage is 84.79%. Comparing base (
5566dd7
) to head (2e6656e
). Report is 12 commits behind head on main.
@coderabbitai review
The updates introduce several new functionalities to the mito2
project, focusing on handling bulk data. Key changes include adding new error variants, enhancing memtable capabilities with bulk operations, defining structures for encoding/decoding bulk parts, and modifying the visibility of certain struct fields. These updates aim to improve data handling efficiency, particularly for bulk operations, and standardize error handling.
File(s) | Change Summary |
---|---|
src/mito2/src/error.rs |
Added UnsupportedOperation variant to Error enum and mapped it to StatusCode::Unsupported . |
src/mito2/src/memtable.rs , .../memtable/partition_tree.rs , .../time_series.rs |
Introduced BulkPart struct and write_bulk method for memtables, with placeholders for unsupported bulk writes in some memtable implementations. |
src/mito2/src/memtable/bulk.rs , .../memtable/bulk/part.rs |
Implemented BulkMemtable and bulk part handling with encoding and decoding functions. |
src/mito2/src/memtable/key_values.rs |
Modified visibility of mutation field in KeyValues and added KeyValuesRef struct for iterating key-value pairs. |
src/mito2/src/row_converter.rs |
Added new_with_primary_keys method to McmpRowCodec for initializing codecs with primary keys. |
src/mito2/src/sst.rs |
Added functions to generate Arrow schema for Parquet storage and define internal fields. |
src/mito2/src/sst/parquet.rs |
Made format module public. |
/test_util/memtable_util.rs |
Added write_bulk method to EmptyMemtable . |
new in src/mito2/src/memtable/bulk/part.rs |
Introduced bulk part encoding/decoding, mutation to Arrow record batch conversion, and various utility functions. |
Amid the code and bytes we tread, Bulk data flows where once we dread. Memtables wrote with eager haste, Errors now with proper grace.
BulkPart
stands robust and tall, Encoding mutations, catching all. Cheers to progress, lines so neat, A coder's job is now complete! 🐇✨
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
PTAL @evenyag @WenyXu @waynexia
@WenyXu PTAL
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
Introduce bulk memtable encoder/decoder.
Checklist
Summary by CodeRabbit
New Features
BulkPart
,BulkPartMeta
, andBulkPartEncoder
.write_bulk
to write bulk data into the memtable.BulkMemtable
for handling bulk load operations.Enhancements
UnsupportedOperation
to the error handling to indicate unsupported operations.McmpRowCodec
with a new method to initialize with primary keys.Bug Fixes