search

apache / rocketmq-flink

RocketMQ integration for Apache Flink. This module includes the RocketMQ source and sink that allows a flink job to either write messages into a topic or read from topics in a flink job.
https://rocketmq.apache.org/
Apache License 2.0
139 stars 88 forks source link

[#35] Use `LitePullConsumer` model instead of default pull consumer #46

Closed yiduwangkai closed 1 year ago

yiduwangkai commented 1 year ago

close #35 Use DefaultLitePullConsumer model instead of default pull consumer.

Tips Thank you very much for contributing to Apache rocketmq-flink. What is the purpose of the pull request *solve some code about using defaultPullConsumer api

Brief change log (for example:)

*Modify RocketMQConfig.java RocketMQSourceFunction.java RocketMQSourceEnumerator.java RocketMQPartitionSplitReader.java RocketMQSourceTest.java

yiduwangkai commented 1 year ago

there are some problems about this code, like home to Use defaultlitPullConsumer to get offset and perform offset self-management

codecov-commenter commented 1 year ago

Codecov Report

Merging #46 (3f5fbe4) into main (90b00be) will increase coverage by 0.11%. The diff coverage is 5.15%.

@@             Coverage Diff              @@
##               main      #46      +/-   ##
============================================
+ Coverage     28.50%   28.62%   +0.11%     
  Complexity      164      164              
============================================
  Files            62       62              
  Lines          2529     2536       +7     
  Branches        269      269              
============================================
+ Hits            721      726       +5     
- Misses         1741     1743       +2     
  Partials         67       67
Impacted Files Coverage Δ
...g/apache/rocketmq/flink/legacy/RocketMQConfig.java 0.00% <ø> (ø)
.../rocketmq/flink/legacy/RocketMQSourceFunction.java 9.80% <0.00%> (+0.24%) :arrow_up:
...g/apache/rocketmq/flink/source/RocketMQSource.java 0.00% <0.00%> (ø)
...nk/source/enumerator/RocketMQSourceEnumerator.java 0.00% <0.00%> (ø)
...nk/source/reader/RocketMQPartitionSplitReader.java 0.00% <0.00%> (ø)
.../apache/rocketmq/flink/common/RocketMQOptions.java 98.41% <100.00%> (+0.05%) :arrow_up:
...ource/table/RocketMQDynamicTableSourceFactory.java 68.18% <100.00%> (+0.73%) :arrow_up:
...mq/flink/source/table/RocketMQScanTableSource.java 27.27% <100.00%> (+0.95%) :arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

SteNicholas commented 1 year ago

Thanks for @yiduwangkai updates. Could you please resolve the conflicts and squash the commits?

zhouxinyu commented 1 year ago

Thanks, @yiduwangkai for your contributions. I left some comments, but I'm not familiar with rocketmq-flink, so please @SteNicholas help have a look.

zhouxinyu commented 1 year ago

Since not all broker versions support lite pull, we need to clarify it in our doc.