search

flink-extended / flink-remote-shuffle

Remote Shuffle Service for Flink
Apache License 2.0
191 stars 56 forks source link

Support to use StatefulSet + Pvc to deploy the shuffle service #66

Open Aitozi opened 2 years ago

Aitozi commented 2 years ago

In the current kubernetes mode, the rss use the hostPath or the emptyDir to store the shuffle data. But, In our company, the local disk resource is very rare and can not meet the storage requirements and It's not the complete storage computing separation architecture I think. I proposal to support to deploy the rss in the way of StatefulSet + PVC , So that we can leverage the cloud storage to store the shuffle data. what do you think of this ? cc @wsry

wsry commented 2 years ago

@Aitozi Thanks. I think that is a good proposal. Would you like to prepare a PR that?

Aitozi commented 2 years ago

Yes, I will prepare a PR for it ASAP and will ping you for review, thanks