Open-Xchange cannot be deployed

[agowa]

Hi, can you please provide more details of how your environment looks like? When I run cat kubernetes.yaml | envsubst | kubectl apply -f -, it fails to deploy successfully. The syntax of the kubernetes.yaml is said to be wrong.

I tried with these environment variables set, but it still fails to start.

export SERVICE_NAME=openxchange
export TZ=utc
export REGISTRY_URI=instantlinux
export DNS_SERVERS=1.1.1.1
export PATH_ADM=/tmp/adm
export PATH_SHR=/tmp/shr
export DOMAIN=mydomain.com

Also I don't see any database server within the kubernetes file, am I right to assume, that it is already integrated into one of the images?

[instantlinux]

• Make sure all the variables mentioned in the file are defined (I've tried to set reasonable defaults in Makefile.vars)
• If you're not using the authelia multi-factor auth you can remove the totp ingress definition from the kubernetes file; likewise you can remove the certmanager annotation if you're not using Let's Encrypt.
• You need to preconfigure a database (separately) and provide credentials with appropriate GRANTs. The mariadb-galera kubernetes file is provided with examples for clustering (as instance db00 in Makefile.instances) and non-clustered single instance (as instance db03). If you create a DNS entry for db03.(your domain) and then invoke make db03 from k8s directory you'll get the single-instance db.

[agowa]

I don't see any MAkefile.vars, so can you please explain how you expect people to apply the configuration? I only want to deploy the open-xchange without wordpress, gitlab, etc...

I think I'm just doing it the wrong way.

[agowa]

It's still falling for me. I uploaded the exact steps to reproduce my error in a clean environment: https://github.com/agowa338/deployment-openXchange

I receive this error:

--Deploying docs--
service/docs created
deployment.extensions/docs created
Error from server (Invalid): error when creating "STDIN": Ingress.extensions "docs-ingress" is invalid: [spec.rules[0].host: Invalid value: "docs.": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[0].hosts: Invalid value: "docs.": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]
Error from server (Invalid): error when creating "STDIN": Ingress.extensions "docs-totp" is invalid: [spec.rules[0].host: Invalid value: "docs.": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[0].hosts: Invalid value: "docs.": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')]
Makefile:68: recipe for target 'docs' failed
make: *** [docs] Error 1

[instantlinux]

Thanks for these details! I'll come up with a response when I have time. As noted, you can comment out the totp resource definition. (Unfortunately kubernetes open-source maintainers have opted not to parameterize their yaml definitions the way, say, CloudFormation does--so there are various projects like helm and ksonnet to templatize them, but until the industry develops a robust standard I'm not yet planning to package these images for kubernetes with anything but yaml resource definitions with a minimalist approach to variable substitution.)

[instantlinux]

I pushed an update that adds a default value for DOMAIN, and parameterizes OX_ETC_READONLY. That'll break your setup.sh script slightly (look at the revised README to see the change) but should make the service come up. I'll be happy to make more adjustments as you run into them; I appreciate your help with making it easier for the next user who wants to use Open Xchange.

[agowa]

I identified some more problems:

1. The secrets are created in the wrong namespace, they are expected to exist within the worker namespace, but are created within the default one.
2. I had the database only listening on localhost.
3. The /var/lib/docker/k8s-volumes directory and its subfolders share and admin are missing and need to be created manually
4. I did not install make, (only a bug within the script, so not relevant for this issue)

Current state:

The container is launched and does not fail to be started. It is still not working, but now at least the container itself is failing and not the engine launching the container. Now the error I receive is about the database not being found (as I'm missing the dns record, will look into it on monday). What I also discovered while debugging, if I enter the openxchange container, it is not able to resolve dns queries, a apt update for example fails as it cannot resolve.

[instantlinux]

See if the updates pushed out earlier today handle the issues you've raised (I didn't address one of them: the share and admin volumes are created using the documented make persistent-dirs target in k8s directory). Have you been able to get the service working?

[agowa]

Have you been able to get the service working?

Not until now, I'm currently having dns issues from within the containers. They throw the error: db00.mydomain.com: forward host lookup failed: Host name lookup failure : Resource temporarily unavailable, even though an nslookup for that domain performed on the host works fine.

[instantlinux]

I noticed in your setup.sh that you're using a VM to run k8s. I had horrendous problems with packet loss (look at the output of ifconfig on the VM running coredns to see if there are errors) when running kubernetes on Virtualbox. The workaround that I've done is to set this in the template.spec section of k8s yaml:

      dnsConfig:
        nameservers: [ $DNS_SERVERS ]
        options:
        - name: use-vc
        - name: ndots

where $DNS_SERVERS is an environment var containing a comma-separated list of IP addresses of your local DNS server(s). To confirm, look at /etc/resolv.conf within your launched container.

[agowa]

I'm not using a VM --vm-driver=none, I'm using minikube with docker, inside of an openstack instance. Anyway after adding that peace of code, the container is unable to resolve any dns name.

[instantlinux]

The above might still help; otherwise you'll have to go through one of the troubleshooting guides like Debugging DNS Resolution. This is what my cluster reports for /etc/resolv.conf:

# kubectl exec openx-container cat /etc/resolv.conf
nameserver 10.96.0.10
search instantlinux.svc.cluster.local svc.cluster.local cluster.local ci.net
options ndots use-vc

You can add nslookup to the container by doing:

$ kubectl exec -it openx-container bash
# apt-get update && apt install -y dnsutils
# nslookup
> db00
Server:     10.96.0.10
Address:    10.96.0.10#53

Name:   db00.instantlinux.svc.cluster.local
Address: 10.244.4.212

[instantlinux]

Inactive issue, reopen if there are any lingering questions.