Closed zhongzc closed 2 days ago
[!NOTE]
Reviews paused
Use the following commands to manage reviews:
@coderabbitai resume
to resume automatic reviews.@coderabbitai review
to trigger a single review.
The changes to the Puffin repository revolve around enhancing the PuffinManager by introducing directory support and data compression mechanisms. This includes updating reader and writer structures, implementing a cache manager, and adding comprehensive test cases. These modifications ensure efficient file handling, caching, and compression capabilities within the Puffin system.
File(s) | Change Summary |
---|---|
src/puffin/src/file_format/reader.rs , .../fs_puffin_manager/reader.rs , .../fs_puffin_manager/writer.rs |
Updated comments for clarity, added data compression comment, and renamed structs/functions to establish uniform naming. |
.../puffin_manager/cached_puffin_manager/cached_puffin_manager.rs , .../cache_manager/moka_cache_manager.rs |
Introduced CachedPuffinManager , removed DELETE_QUEUE_SIZE constant, implemented new struct methods and trait extensions. |
src/puffin/src/puffin_manager/tests.rs |
Added new test functions for CachedPuffinManager covering directory and file operations with various compression codecs. |
src/puffin/src/puffin_manager.rs |
Introduced new test module and updated BlobGuard trait to include the Unpin requirement for Reader . |
src/puffin/src/puffin_manager/fs_puffin_manager.rs |
Introduced FsPuffinManager , providing filesystem-oriented puffin data management with reading and writing functionalities. |
sequenceDiagram
participant User
participant CachedPuffinManager
participant Cache
participant FsPuffinWriter
participant FsPuffinReader
User->>CachedPuffinManager: Put directory
CachedPuffinManager->>FsPuffinWriter: Write data with compression
FsPuffinWriter->>Cache: Store compressed data
User->>CachedPuffinManager: Get directory
CachedPuffinManager->>Cache: Retrieve data
Cache->>FsPuffinReader: Fetch compressed data
FsPuffinReader->>cachedPuffinManager: Read data
CachedPuffinManager->>User: Return data
Objective | Addressed | Explanation |
---|---|---|
Support footer lz4 compression (#4193) | ✅ | |
Introduce puffin manager trait (#4193) | ✅ | |
Introduce writer for puffin manager (#4193) | ✅ | |
Introduce cache manager (#4193) | ✅ | |
Introduce reader for puffin manager (#4193) | ✅ |
In a warren of code so tight,
Puffin files now take flight,
With caches swift and data compressed,
Our changes stand the test,
Bringing joy, both day and night.
🐇✨
[!TIP]
Early access features: enabled
We are currently testing the following features in early access: - **OpenAI `gpt-4o` model for code reviews and chat**: OpenAI claims that this model is better at understanding and generating code than the previous models. We seek your feedback over the next few weeks before making it generally available. Note: - You can enable or disable early access features from the CodeRabbit UI or by updating the CodeRabbit configuration file. - Please join our [Discord Community](https://discord.com/invite/GsXnASn26c) to provide feedback and report issues. - OSS projects are currently opted into early access features by default.
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
Attention: Patch coverage is 98.82629%
with 5 lines
in your changes missing coverage. Please review.
Project coverage is 84.73%. Comparing base (
e531326
) to head (aaa5ea1
). Report is 3 commits behind head on main.
https://github.com/GreptimeTeam/greptimedb/actions/runs/9755006232/job/26922900846?pr=4240
Is this related to https://github.com/GreptimeTeam/greptimedb/pull/4243? @WenyXu
thread '<unnamed>' panicked at tests-fuzz/targets/fuzz_insert_logical_table.rs:307:35:
fuzz test must be succeed: 0: Failed to execute query: CREATE TABLE `CupidiTaTE`(
ts TIMESTAMP(3) TIME INDEX,
val DOUBLE,
`eT` STRING,
`VoLUPTaTe` STRING,
`ASPerNatur` STRING,
`ExPLiCabo` STRING,
PRIMARY KEY(`VoLUPTaTe`, `ASPerNatur`, `eT`, `ExPLiCabo`)
)
ENGINE=metric with ("on_physical_table" = "doLORuM");, at tests-fuzz/targets/fuzz_insert_logical_table.rs:256:14
1: Database(MySqlDatabaseError { code: Some("HY000"), number: 1815, message: "Region `4647154614272(1082, 0)` already exists" })
https://github.com/GreptimeTeam/greptimedb/actions/runs/9755006232/job/26922900846?pr=4240
Is this related to #4243? @WenyXu
thread '<unnamed>' panicked at tests-fuzz/targets/fuzz_insert_logical_table.rs:307:35: fuzz test must be succeed: 0: Failed to execute query: CREATE TABLE `CupidiTaTE`( ts TIMESTAMP(3) TIME INDEX, val DOUBLE, `eT` STRING, `VoLUPTaTe` STRING, `ASPerNatur` STRING, `ExPLiCabo` STRING, PRIMARY KEY(`VoLUPTaTe`, `ASPerNatur`, `eT`, `ExPLiCabo`) ) ENGINE=metric with ("on_physical_table" = "doLORuM");, at tests-fuzz/targets/fuzz_insert_logical_table.rs:256:14 1: Database(MySqlDatabaseError { code: Some("HY000"), number: 1815, message: "Region `4647154614272(1082, 0)` already exists" })
No, it seems something else. Please trigger test again, and I’ll check the failed action ASAP.
@coderabbitai pause
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
close #4193
What's changed and what's your intention?
as @coderabbitai
Checklist
Summary by CodeRabbit
New Features
CachedPuffinManager
for managing cached readers and writers.FsPuffinManager
for handling puffin data in the filesystem.CachedPuffinManager
with various compression codecs.Bug Fixes
Refactor
CachedPuffinReader
toFsPuffinReader
.CachedPuffinWriter
toFsPuffinWriter
.Documentation
Chores
DELETE_QUEUE_SIZE
constant.