terrymanu
commented
3 years ago I post the Chinese translation too:
ElasticJob 后续设计规划
- 产品定位
ElasticJob 目前是基于定时任务的分片调度中间件，在 ElasticJob-Cloud 中增加了资源治理的能力。 未来的 ElasticJob 希望将功能划分为独立的三个部分：任务触发、资源管理、任务治理。
任务触发是必选模块，其他两个模块是可选的。只有任务触发，相当于降级为 QuartZ。 任务触发 + 任务治理可以理解为 ElasticJob 的现状。在任务触发的同时，增加分布式治理和任务分片的能力，未来的基于有向无环图的任务编排也属于任务治理模块。 任务触发 + 资源治理可以理解为类似于操作系统的任务调度机制。在任务执行时增加资源的管控，在资源不足的情况下将任务排队，资源管控可以对接 Kubernetes 和 Apache Mesos。目前的 ElasticJob-Cloud 则是任务触发 + Mesos 资源治理实现方式。
任务触发 + 任务治理 + 资源治理是 ElasticJob 未来的全部能力，而 ElasticJob-Lite 和 ElasticJob-Cloud 则仅仅是部署形态不同。
- 架构设计
ElasticJob 希望采用可插拔架构设计，将功能模块通过 SPI 动态织入现有架构体系。针对于当前设计的三大功能模块，其可插拔设计分别是：
任务触发：目前是基于 CRON 表达式和一次性调度两种触发形式。希望调整为调度 SPI + 调度实现（CRON，One-Off，其他...）。 资源治理：目前没有独立的资源治理抽象层。希望未来在增加资源治理 SPI 的同时，增加基于Apache Mesos、Kubernetes 和 NoDep 的资源治理实现模块。 任务治理：目前只有任务分片和高可用治理两部分。希望未来提供任务治理 SPI，并将分片和高可用作为其实现模块，并在此基础上增加 DAG 治理能力。任务治理的各个模块相互隔离且可叠加。
- ElasticJob-Lite 和 ElasticJob-Cloud 调整
ElasticJob-Lite 和 ElasticJob-Cloud 调整为仅仅是部署形态不同。 ElasticJob-Lite 采用无中心化的 jar 部署形态，提供 spring 等框架的接入层，适用于轻量级应用。 ElasticJob-Cloud 采用中心化的 server 部署形态，适用于集中化的作业云管平台。
- 模块规划
任务触发：
- Trigger API
- Trigger SPI
- Trigger Kernel
- CRON Trigger
- One-Off Trigger
- Customized Trigger
资源治理：
- Resource Management API
- Resource Management SPI
- Resource Management Kernel
- NoDep Resource Management
- Mesos Resource Management
- Kubernetes Resource Management
任务治理：
- Job Governance API
- Job Governance SPI
- Job Governance Kernel
- Sharding Job
- HA Job
- DAG Job
产品形态：
- ElasticJob-Lite 架构调整
- ElasticJob-Cloud 架构调整
- ElasticJob-Cloud 剥离 Mesos
turnhead
topsnowwolf
TeslaCN
skaic
krystalics
The current ElasticJob is an distributed job scheduler middleware and support resource management in ElasticJob-Cloud. The new ElasticJob may split the feature into 3 independent modules: Job Trigger, Resource Management and Job Governance.
Job Trigger is required module, other two are optional. It just likes a standard QuartZ if using Job Trigger only. Job Trigger + Job Governance can be understood as current ElasticJob. There are sharding and HA features in scheduled job, we can add job orchestration such as DAG into this module in future. Job Trigger + Resource Management can be understood as mechanism of operation system's scheduler. It will balance jobs and resources, if resources is not enough, the job should be in queue. Resource Management can integrate with Kubernetes and Apache Mesos. The current ElasticJob-Cloud is Job Trigger + Resource Management (with Apache Mesos only). Job Trigger + Job Governance + Resource Management is all abilities for ElasticJob in future. ElasticJob-Lite and ElasticJob-Cloud is for different deploy architect only.
ElasticJob hope to build a
Flexible & Embeddable & Extensiblearchitect, which can permit developers weave their own implementation via SPI. For the 3 main modules, the details are:Job Trigger: Based on CRON expression and One-Off job for now. It is better to adjust as Trigger SPI + Trigger Impl (CRON, One-Off and so on). Resource Management: There is no independent resource management domain abstraction for now. It is better to add Resource Management SPI, and add Apache Mesos, Kubernetes and NoDep's implementation in the meanwhile. Job Governance: There are sharding and HA modules for now. It is better to add Job Governance SPI, and adapt the current modules as sub-modules, and add DAG module in future. The sub-modules of Job Governance should be overlying and isolated each other.
The only different of ElasticJob-Lite and ElasticJob-Cloud are deploy architect. ElasticJob-Lite is centre-less jar with spring integration, it is suitable for lightweight applications. ElasticJob-Cloud deploy a clustered job-server , it is suitable for job cloud management platform.
Job Trigger
Resource Management
Job Governance
Production