search

airyhq / airy

💬 Open Source App Framework to build streaming apps with real-time data - 💎 Build real-time data pipelines and make real-time data universally accessible - 🤖 Join historical and real-time data in the stream to create smarter ML and AI applications. - ⚡ Standardize complex data ingestion and stream data to apps with pre-built connectors
https://airy.co/docs/core
Apache License 2.0
371 stars 44 forks source link

Deploy the helm charts #1014

Closed ljupcovangelski closed 3 years ago

ljupcovangelski commented 3 years ago

As we have the helm charts templates in our repo (for a particular version, bundled in the CLI), we would want to generate all the manifests with cli and apply them to the Kubernetes cluster.

This needs to be part of the airy create command and we would need to use some helm libraries, so that we don't depend on the helm binary.

lucapette commented 3 years ago

I would also close this as there's already an issue for this?

ljupcovangelski commented 3 years ago

I think this deserves a separate issue as it is a significant part of getting rid of the ./scripts/bootstrap.sh script.

We need to find a way to do helm template + kubectl apply or helm install directly from the go code inside the airy create command.

lucapette commented 3 years ago

@ljupcovangelski I think we agreed on calling helm via a go lib from our code. The "challenge" is to fetch the helm charts from github (at the corresponding version). I do stand by the point that this is "just" part of #1017

pascal-holy commented 3 years ago

The Helm way of doing this is to host a Helm Repo which is just an HTTP server that hosts an index.yaml with a description of the different charts and a zip file containing them.

It can be hosted on Github pages .

The benefit is that we can directly refer to the charts of the corresponding version in the helm install --repo --version command (or the Helm lib equivalent of it.

lucapette commented 3 years ago

@pascal-airy I like that quite a bit... but I can't picture how well it plays with release cycle and mono repo. But I think we should push to figure this out as this is much more elegant than us trying to download files and do it "manually"

I think this issue should also mention treafik as we're assuming we will install it in this step (and configure it as well?)

ljupcovangelski commented 3 years ago

This issue is indeed part of #1017. My preference would be to keep it as a separate just for better readability and discussion. But we can also merge to that one. I consider it only for the go lib part on how do we deploy the charts and not for which charts should be added (we have a task for the traefik part in the umbrella issue)

chrismatix commented 3 years ago

@paulodiniz and I had a discussion on how we interface with Helm:

It's harder to maintain direct interaction with Helm golang APIs, which is why we like the appeal of @ljupcovangelski's solution of using a Kubernetes job with an image that contains Helm and the correct chart version.

TODO: