Organizing Tutor.. Bringing OpenEdx to the Modern Cloud

[travlr]

Tutor and @regisb have done a wonderful job of containerizing OpenEdx and now beginning to enhance scalability and the deployment/admin process by incorporating Kubernetes.

At the moment, the Kubernetes effort is, as expected, to simply be able to deploy OpenEdx to a Kubernetes enabled infrastructure.

Obviously, as this stage is met, the next considerations must be towards towards reliable, scalable databases. This would be accomplished through using StatefulSets instead of a regular Deployment manifest. I know that I am just re-iteritating what @regisb has in mind here.

The next considerations would be incorporating Helm and Istio. Helm, the package manager for kubernetes deployments, makes the manifest generation process and package management easier. Istio is a layer on top of kubernetes that provides all sorts of features needed for a truly scalable and secure micro-service orientated architecture.

Eventually, I would love to see the entire OpenEdx architecture to be re-considered and broken up into atomic micro-services.

This work has already been started by @regisb and I could easily see a small community evolving around Tutor.. possibly with collaboration from the broader openedx developer community.

What are your thoughts @regisb? If you find this favorable to your desires, then maybe a chat channel would help the conversation's fluidity.

Thanks for your work on Tutor and looking forward to contributing.

Steps to reproduce

N/A

Unexpected behavior

N/A

Additional info (IMPORTANT)

N/A

[regisb]

Thanks for starting this conversation @travlr! There is food for thought here :) To be 100% honest, I'm a relative noob in the Kubernestes ecosystem, so I don't really know what the good practices are. Let me find some documentation on StatefulSet and Istio and I'll get right back to you ;) But you are absolutely right, one of my goals is to create a robust and scalable (aka: highly available), K8s-based Open edX platform.

This is a little outside the scope of this conversation, but I'm interested in learning how you use Tutor. Maybe you can send me a short email describing what you do?

[travlr]

Hi @regisb, I have a good knowledge of kubernetes, but there's always more to learn ;-).

Take a look at the already stable helm charts for the various dependencies. Imho, I think that it would be easiest to start utilizing helm right away and also get the benefits of properly implemented, difficult dependencies like the databases, elasticsearch and rabbitmq.

https://github.com/helm/charts/tree/master/stable

[frob]

I don't see the need for a chat channel at the moment. The issue queue is a better place to hold discussions such as this.

[travlr]

Hi @regisb @frob. Thank you for your considerations.

The project that I am currently working on has an immediate requirement to get what I've proposed above implemented.. with the exception of re-architecting the openedx code base into atomic microservices ;-) .

I also read @regisb, that you are announcing Tutor at the openedx conference. I would love to help you get Tutor to where you want it to be.

I am working on the helm implementation and will post a link to the repo. I'm creating deploy/helm adjacent to deploy/k8s.

[DiegoGallegos4]

@travlr How would you go on regarding stateful components (mysql, mongo, elasticsearch...)? Wouldn't it be better to provide external services for this (e.g aws rds) as a 1st step?

What I mean by external services is do not add stateful components on the deployment to the cloud provider but instead only provide the configuration of this externals services (url, host, port, etc).

I do not know much about deploying databases in k8s but it'll be great to layout a roadmap so more people can jump in into helping move this forward.

[regisb]

@travlr Thanks for the offer! I appreciate it. For now, Kubernetes deployment is pretty much an alpha feature in Tutor, but I heard some people have had more success (side glance at @lgfa29). A one-click helm install would definitely be a plus. However, I don't want tutor users (nor myself) to be bound to a single cloud provider. Also, Tutor should be "all batteries included", so that users don't have to rely on external services. My aim is thus to include all stateful services: mysql, elasticsearch, mongodb, rabbitmq, memcached.

However, I understand that some users would rather use hosted solutions, such as RDS. This is the topic of issue #114, and we are going to take care of it, but it's a separate issue from this one.

[travlr]

@travlr Thanks for the offer! I appreciate it. For now, Kubernetes deployment is pretty much an alpha feature in Tutor, but I heard some people have had more success (side glance at @lgfa29).

I didn't find this work in his repos.. can you provide a direct link

A one-click helm install would definitely be a plus. However, I don't want tutor users (nor myself) to be bound to a single cloud provider.

helm cli depends on kubectl config current-context. So it is independant of deployment and lets kubectl config handle the cluster to operate on.

helm is the offical kubernetes package manager for the CNCF.. It is infrastructure agnostic.. as is the goal of CNCF/kubernetes/istio in general.

Also, Tutor should be "all batteries included", so that users don't have to rely on external services.

I saw that you are moving from make to a bespoke comand interface (fwiw, my favorite for this consideration is Go lang and the Cobra/Viper packages).

You did such a nice job with your make files, imho.. and provided great ease with optional considerations and workflows, etc.. So naturally, the desire to customize installations as @DiegoGallegos4 is already provided via configuration prior to deployment event.

My aim is thus to include all stateful services: mysql, elasticsearch, mongodb, rabbitmq, memcached.

So, as I mentioned, the helm charts available for these dependencies are already well considered for HA and only require most-minimal configuration for deployment..

Helm is a great resource for cutting down the boilerplate required to deploy services.. and there is a great wealth of charts already created that cuts down on duplicated efforts as well.

I know I'm just re-iterating what is already understood by some, but I'm also stating this for those that are new to kubernetes/helm/istio

My thoughts are that helm should be implemented first to take advantage of these, available, verified charts that will deploy the needed dependencies (databases, etc) with assurance of there quality. And of course as a separate consideration, this will be an option decided by the user at configuration time.

At the same time I am creating the deploy/helm repo, I am also including the manifests needed to enable Istio as an option.. Istio does not use an ingress-controller per se, but devised its own edge-proxy called a "Gateway". So, an optional consideration for the user is whether or not they have an Istio enabled cluster (which of course could be automated) and whether their deployment is destined for standard kubernetes or istio-enhanced kubernetes.

and.. I'll have a link to my repo branch posted here soon.

[travlr]

I created a new branch in my repo called helm , where there is a new deploy/helm directory. At time of this push, only initial scaffolding is commited. This initial implementation will be a standard k8s install w/out Istio considerations.

[DiegoGallegos4]

@travlr Is there any make command to test it? or directly through helm cli?

[travlr]

Hi @DiegoGallegos4 .. Its just scaffolding at the moment.

[regisb]

@travlr just wanted to say that I did not forget you, despite my lack of answer! I was busy fixing stuff in Tutor after the Great Big Rewrite. I hope to review your branch soon.

[travlr]

@regisb yeah, my apologies too, I wanted to be more engaging as well, I got to a point with helm deployment a while a go (got stuck w/ mongo authentication) and had to put it aside for a little while. Currently, I'm working with your 'local' deployment and trying to overcome struggling with hawthorn's theming.. I hope Ironwood goes to all bootstrap :-P .. what a pain in the ass with all the different mechanisms combined.. and lack of any useful documentation.. but I digress ;-)

[JuanSeBestia]

I am really fascinated with this issue, in my team we are going to start doing exactly what @travlr wants.

unfortunately I did not find the @travlr 's repo

If you have any suggestion don't hesitaste to contact me :)