Create Kubernetes Operator

[javib51]

Create a Kubernetes Operator to allow easy setup for appwrite stack in any Kubernetes cluster.

As discussed in issue #24.

[javib51]

My idea is to start with a MVP of the operator and add more functionality afterward.

For now, development setup:

• one instance of each container
• redis, influxdb and mariadb are included

What do you think @eldadfux?

[eldadfux]

We would probably need Traefik container as well. ClamAV is going to be optional in next release (0.6.2).

You can use this compose file as a source of truth: https://appwrite.io/docker-compose.yml

[javib51]

@eldadfux thanks! I'm going to follow it.

Another thing. For now, Im going to develop it in a private repo. Whenever it's ready, we can decide where to place it.

[eldadfux]

Sounds great, feel free to reach out on Discord in you want to chat more about this.

[eldadfux]

hey @javib51, any update on this one?

[alonsomoya]

+1

[kodumbeats]

Might be able to leverage Kompose to convert docker-compose.yml -> helm chart (tackling #24), and according to RedHat, the Operator SDK knows what to do:

One of the Operator SDK options for generating an Operator project includes leveraging an existing Helm chart to deploy Kubernetes resources as a unified application, without having to write any Go code.

I'll update if I can get it to work.

[mcchin]

I have been following Appwrite for a while now, looking forward to setup in k8s environment for my personal projects. Armed with basic knowledge of k8s, I thought to give it a shot this weekend to at least able to deploy Appwrite on GKE, and see if I could at least get the admin dashboard up.

Looking through threads here at Github and Discord, I tried Kompose (ver 1.21.0) and as mentioned in previous threads on discord it won't run outright (pods keep crashing), unless I exposed some of the deployments as a service (redis, influxdb, telegraf, clamav), and update the PVC with r+w permission for the config and certificate volume. (There is probably a correct way for this, but for now I just want to get it up)

So with that alone and with default env variables, I could do a portfoward and access Appwrite dashboard locally (e.g. http://127.0.0.1:8080), and able to signup, login, and start creating projects.

Although at this point some of the pods will still keep restarting (probably need to work the configs a bit more for request-catcher, clamav, adminer). But the biggest blocker so far I could see is Cloud Function as looking at the source codes like:

...
Console::execute('docker pull '.$environment['image'], '', $stdout, $stderr);
...
$exitCode = Console::execute("docker exec ".\implode(" ", $vars)." {$container} {$command}...
...

I guess needs to setup in a way like docker in a docker or able to execute docker from the Node in k8s. Basically what I am trying to get at is that this is probably not going to be straightforward.

But I guess I probably don't need Cloud Function for my personal projects and see how much further I could get to next weekend.

Edit1: k8s yaml that I am using https://gist.github.com/mcchin/bb0974f9f29ca39af338d56ffc033216 Edit2: Is there a tutorial on how to setup Appwrite that is scalable with just docker on cluster of machines?

[dimm0]

I'm looking at deploying appwrite on our kubernetes cluster, and dependency on docker is a warning for me so far. Docker is going to be deprecated in kubernetes soon, and also it's a security vulnerability to expose it's socket to containers. Why is it required at all?

I might contribute to the operator if there's something available already

[kodumbeats]

@dimm0 We mount the Docker socket only in our functions worker to execute cloud functions in standalone containers. We're aware of the security concerns with the Docker socket and are still engineering a solution as we build our own hosted service.

[tombailey]

@mcchin this is a really helpful start, thanks! Your gists seem to be using some of the local dev tools instead of production versions; e.g. maildev instead of appwrite/smtp. That might explain some of the issues you saw. I am looking into fixing that. If I can make some good progress, I will try to provide some updated templates to reflect the latest version

[eldadfux]

@tombailey note that since v0.7.x we don't use appwrite/smtp as the default SMTP configuration. By default we use no SMTP settings at all, and we added messages, and disabled SMTP related functionality in the Appwrite console in such case (where no SMTP was set). This was done to avoid confusion as appwrite/smtp requires a lot more config in order to work properly and pass SPAM filters.

[tombailey]

Similar to @mcchin I was able to load the dashboard, sign up and create projects but not much more. File uploads seemed to fail because ClamAV had permission issues doing lstat on uploaded files. I configured SMTP using sendgrid but didn't receive any emails or see any activity in the sendgrid dashboard. It can take a long time to debug issues like this so I have stopped for now.

The k8s deployment files here: https://gist.github.com/tombailey/5f01c4faaec62757b2954ec15f28f49c

It is setup to use secrets for maria/mysql passwords and sendgrid API key. Ideally, a secret should be added for _APP_OPENSSL_KEY_V1 too

[loftwah]

@kodumbeats I see why this spun off, I don't think we need a seperate issue for the Helm Chart, because it would be covered here.

[codeagencybe]

Hello

Any recent update/progress on this? I'm also looking into appwrite in Kubernetes. I still see a lot Docker at the moment, everybody is aware this is changing (or better yet already changed) to containerD.

On my local machine, I'm using Rancher Desktop which uses 1:1 replacements for Docker to Nerdctl and KIM. Links below for interested.

https://github.com/containerd/nerdctl https://rancherdesktop.io/ - https://github.com/rancher-sandbox/rancher-desktop

[gewenyu99]

This issue has been open for a while, but I wanted to provide an update.

First of all, thank you to all involved in the discussion for contributing. It's super important to hear your ideas, we make much of our decisions based on what we hear on GitHub.

This is currently not something we're actively working on, because we're focused on other efforts to make Appwrite more feature complete, reliable, and performant. That said, this is still a strong interest area that we plan to explore eventually.

Thank you to each and every one of you for your patience ❤️

[rg42]

Did anyone try deploying appwrite in a docker image (or docker in docker)? It's at least something until appwrite is officially available on kubernetes

[nesterow]

Did anyone try deploying appwrite in a docker image (or docker in docker)? It's at least something until appwrite is officially available on kubernetes

It works with the systemd enabled Sysbox images in pair with a docker:dind job which starts appwrite-executor. Share docker.sock and other mounts for pods and docker containers that run inside the docker pod. You need the right configuration for _APP_EXECUTOR_RUNTIME_NETWORK and the _APP_EXECUTOR_HOST should resolve to the sysbox's docker pod. Also, you have to pass the environment from pod to the docker containers. I use it only for the executor, but there's a chance It works for everything in one pod. I wouldn't recommend it though.

This is the best solution I could figure out. Currently, this setup works locally in the dev. environment - no issues so far.

[stnguyen90]

Related: https://github.com/utopia-php/orchestration/issues/31