Closed paomian closed 1 month ago
[!IMPORTANT]
Review skipped
Auto reviews are disabled on this repository.
Please check the settings in the CodeRabbit UI or the
.coderabbit.yaml
file in this repository. To trigger a single review, invoke the@coderabbitai review
command.You can disable this status message by setting the
reviews.review_status
tofalse
in the CodeRabbit configuration file.
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
Attention: Patch coverage is 74.52019%
with 385 lines
in your changes missing coverage. Please review.
Project coverage is 84.64%. Comparing base (
216bce6
) to head (c34c92f
).
I think we must deliver this PR in v0.9.1, so please take a look @sunng87 @shuiyisong @zhongzc
Incompatible changes added
date
, epoch
processortimestamp
process, which is a combination of date and epoch. It can parse time strings as timestamps or treat numbers as timestamps.time
and epoch
from transfrom and replace them with timestmap
.time
index type~ use time
replace timestamp
index by time
Incompatible changes added
- Delete
date
,epoch
processor- add
timestamp
process, which is a combination of date and epoch. It can parse time strings as timestamps or treat numbers as timestamps.- remove
time
andepoch
from transfrom and replace them withtimestmap
.- add
time
index type
I don't like breaking changes. You can just deprecate the old processors and remove them from docs.
What's the progress of this PR? @shuiyisong @paomian
What's the progress of this PR? @shuiyisong @paomian
There are a few details that need to be changed. Prepare for careful testing.
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
There are over a thousand lines of code that are test data and benchmarks can be ignored.
src/pipeline/benches/data.log
is test data. andsrc/pipeline/benches/processor.rs
is benchmark.timeindex
andtimestamp
are supported to specify the timeindex in the transform.Original pipeline workflow
Current pipeline workflow
In the original method, there is a lot of data allocation and drop, and a lot of hash get operations. By converting the data into a vector, hash get operations are avoided. And the same batch of data share a common intermediate state. After testing, the performance of the same pipeline processing the same data is improved by more than 30%.
Checklist