search

DIRACGrid / diracx-charts

Helm charts for running DiracX
1 stars 17 forks source link

Run k3s on bare metal/VM #107

Open chaen opened 3 months ago

chaen commented 3 months ago

K3s is the recommended way to run diracx for installation without an existing kubernetes like cluster (k8s, openshift, magnum, etc). However, the problem will be to expose that cluster to outside. That involves DNS/routing and also certificate handling.

For the DNS aspect, the commonly found solution involves metallb and external_dns

For certificates handling, I need to investigate. cert-manager does not allow static certificates.

fstagni commented 2 weeks ago

Hi @bertrandrigaud , me and @atsareg are effectively trying to run the diracx chart on bare VMs today, using 3 VMs (Alma9). The cluster will need to be exposed to the outside, and we are running into the issues described above. For the moment (as a poor-man-solution) we moved to NodePort for MySQL and OpenSearch: https://github.com/DIRACGrid/diracx-charts/pull/114 . We will also assign a certificate to each VM. This denies some of the flexibility that we could have from the system, so improvements in this respect would be welcome. Did you have chance to investigate the possible solutions mentioned above?

marianne013 commented 4 days ago

Hi, I just had a quick chat with Simon and he noted that the problems Chris sees were exactly the same as we saw when we tried to deploy kubernetes at Imperial, and that this would always be an issue when transitioning from k3s to proper kubernetes. @fstagni It is my understanding that k3s only needs one machine, not three (though it can). Is there any particular reason you tried it with 3 ? Sorry, I'm still catching up with all of this.

fstagni commented 4 days ago

I tried with both 1 and 3, as 2 is "not recommended" for a reason that is not important right now. I can try with 10 machines but that's not the point. I think we are just discovering issues (maybe not depending by us) while going on, e.g. https://mattermost.web.cern.ch/diracx/pl/j3neiyso7fbqmxoi8rugrg7n9r