search

dromara / CloudEon

CloudEon uses Kubernetes to install and deploy open-source big data components, enabling the containerized operation of an open-source big data platform. This allows you to reduce your focus on underlying resource management and maintenance.
https://www.cloudeon.top/
Apache License 2.0
446 stars 107 forks source link

[Enhancement] Remove HDFS/YARN dependency requirements for Dinky/Trino/DolphinScheduler/Flink #97

Closed leeoo closed 1 year ago

leeoo commented 1 year ago

Is your feature request related to a problem? Please describe. In my bigdata environment, there is no any Hadoop components, I use MinIO as the DFS, and use ZooKeeper for Flink HA, when I deploy Dinky/Trino/DolphinScheduler/Flink on CloudEon, it blocks me, and popup tips "需要提前安装服务：HDFS, YARN".

Actually, HDFS/YARN is NOT the only option for Dinky/Trino/DolphinScheduler/Flink components, for example, we can use Amazon S3, MinIO, Alibaba OSS to replace HDFS, and using ZooKeeper to replace YARN for Flink standalone HA, so these two components shoud NOT be the required option.

Describe the solution you'd like Remove HDFS/YARN dependency requirements for Dinky/Trino/DolphinScheduler/Flink.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

Pandas886 commented 1 year ago

I think it is better to make a new stack(eg: KDP-xx ) to support this issue.