marsupialtail / quokka

Making data lake work for time series
https://marsupialtail.github.io/quokka/
Apache License 2.0
1.14k stars 60 forks source link

Kubernetes support #54

Open marsupialtail opened 1 year ago

marsupialtail commented 1 year ago

This is a long running issue on trying to make Quokka work on Kubernetes.

Currently Quokka works on a Dockerized setup on a Ray cluster, so the leap to Kubernetes is probably not big. We can also refer to how Spark works on Kubernetes: https://spark.apache.org/docs/latest/running-on-kubernetes.html

We can probably get started with assuming that Quokka pods will not get pre-empted. If pods do get pre-empted, Quokka does not tolerate coordinator failures.