[Discuss][Seatunnel-Engine]change queue to disruptor.

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.

https://seatunnel.apache.org/

Apache License 2.0

7.82k stars 1.76k forks source link

[Discuss][Seatunnel-Engine]change queue to disruptor. #3849

Closed liugddx closed 1 year ago

liugddx commented 1 year ago

Search before asking

[X] I had searched in the feature and found no similar feature requirement.

Description

Now the queue we use is BlockingQueue. I want to replace it with Disruptor and test its performance under massive data

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

liugddx commented 1 year ago

I want to try it, so I'm ready to test it. Is this a meaningful issue for Seatunnel ? @CalvinKirs @hailin0 @Hisoka-X @EricJoy2048 @TyrantLucifer @ashulin @ic4y

Hisoka-X commented 1 year ago

I want to try it, so I'm ready to test it. Is this a meaningful issue for Seatunnel ? @CalvinKirs @hailin0 @Hisoka-X @EricJoy2048 @TyrantLucifer @ashulin @ic4y

Great! This is a feature I'm very curious about the result.

TyrantLucifer commented 1 year ago

We can disscuss this in meeting tonight. Prepare the design plan and the necessity of the proposal before the discussion is better.

ic4y commented 1 year ago

I want to try it, so I'm ready to test it. Is this a meaningful issue for Seatunnel ? @CalvinKirs @hailin0 @Hisoka-X @EricJoy2048 @TyrantLucifer @ashulin @ic4y

This is a good idea, and care about resource usage when testing.