[Feature] Introduce seatunnel-backend module

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.

https://seatunnel.apache.org/

Apache License 2.0

7.96k stars 1.8k forks source link

[Feature] Introduce seatunnel-backend module #1573

Open simon824 opened 2 years ago

simon824 commented 2 years ago

Search before asking

[X] I had searched in the feature and found no similar feature requirement.

Description

This issue want to discuss do we need a seatunnel-backend module that supports remote submission of tasks , currently we having to execute the submit command on the server side.

And we can discuss what feature we needed in seatunnel-backend module.

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

ruanwenjun commented 2 years ago

Good idea to me, I hope we can provide seatunnel-backend module or a seatunnel-cli client, that can help user to manage the seatunnel jobs like submit job/ stop job or some other operation, user don't need to extra use flink/spark client to do this work.

And furthermore, if the added module can provide the plugin information is better. Such as how many kinds of plugins have been packaged in the seatunnel jar, and what params is needed of the plugin.

simon824 commented 2 years ago

Thank you for your feedback, I will design the initial version. Also I want to discuss which framework should we choose for providing HTTP services, Jetty or Spring Boot or others, WDYT? @CalvinKirs @wuchunfu @ruanwenjun

simon824 commented 2 years ago

Jetty +1 IMO, jetty is light and springboot is heavy, backend module is more about forwarding work and state acquisition.

ruanwenjun commented 2 years ago

Thank you for your feedback, I will design the initial version. Also I want to discuss which framework should we choose for providing HTTP services, Jetty or Spring Boot or others, WDYT? @CalvinKirs @wuchunfu @ruanwenjun

If we don't need to integrate with other framework, using Jetty is good to me. If we still need to do some database operation, I think use SpringBoot is a better choose.