apache / shardingsphere-elasticjob

Distributed scheduled job
Apache License 2.0
8.15k stars 3.29k forks source link

Support New Sharding Strategy To Run Balanceful On Multiple Servers For Single Sharding Scenario #2461

Open baiyangzhuhong opened 12 hours ago

baiyangzhuhong commented 12 hours ago

Feature Request

For English only, other languages will not accept.

Please pay attention on issues you submitted, because we maybe need more details. If no response anymore and we cannot make decision by current information, we will close it.

Please answer these questions before submitting your issue. Thanks!

Is your feature request related to a problem?

Yes, here is the detail information.

We use elastic job as job framework in our job applications heavily, however our job applications are always running a job in single sharding which means the sharding total count is only 1, and our job applications are always deploying on more than two servers.

You know, there are many job tasks in one job application, some job tasks maybe run quickly and need a little resources, but some job tasks are very heavy, need much of resources and running will be elapsed a lot time. The problem is for a certain job task is always running on a certain server after sharding scheduled, that caused one server would occupy much more CPU and memory for all the heavy tasks are running on, but the other server is more idle for the running on tasks are lightly.

we want the resources like cpu, memory will be used balanced, and not waste some resources just for a standby failover.

Describe the feature you would like.

We want one job task to run balanced in multiple servers.

For example, there is a heavy task that reading file and compare with the DB records then do business handling, and there are 3 servers A, B, C used to deploy on. Currently, the builtin sharding strategy in the framework can only support to split all the jobs and assign them to different server, like using 'ROUND_ROBIN' job sharding strategy type, but the assigned job is sticked with the server, let's say it is assigned to server A, then can't be changed.

However, what we want to achieve is this heavy task can run on multiple servers balanced, like it is firstly running on server A, then the second time it will be running on server B, then the third time it will be running on server C, and the next time it will be running on server A again, and like this in turn.

Solution

The elastic job version what we currently using is 3.0.1, and I already resolve this problem and finish the solution of this feature. I deploy our job applications that using this new feature in our product environment, it works well.

I have searched all the open issues that there is no issue talked about this feature, and also I check the codes above version 3.0.1, eventually the latest master codes, no feature about this.

So I post this ticket and want to discuss here, could this feature be supported in the framework? if it is accepted, I could contribute the codes that I have written to the community.