Publish an image that is ready for scalable cloud deployments

[schroedermatthias]

Hi Folks,

thank you for providing a next cloud container image!

Unfortunately I spent a lot of time to get nextcloud running in my Kubernetes cluster. Unfortunately it's still not running and I would like to consider/discuss the following aspects.

rsync on startup?

As far as I can see, the entrypoint script rsyncs files from the container image to a different location: https://github.com/nextcloud/docker/blob/7bd3b7b6dbd694bab3612a28b17a9767f2339c9e/docker-entrypoint.sh#L177

In my eyes, a containerized application should be self-containing and source code should be ran from the image (ideally stored there in read-only mode, immutable for security reasons).

Can someone explain to me, why this image copies source files (even third party code, like the aws library) to another location before starting? For me, this seems to be an anti-pattern in the world of container images. I read something about upgrading, but I think upgrading in the container world means - not to compare any version.php files residing in a persistent file system - but to change the tag of the image. The application itself (again running from a readonly part of the filesystem) should detect its current version and compare this to the database. Then it has to handle the upgrade process (if required) with lockfiles, leases, migrations, etc. Also from a performance perspective I disagree with the approach to rsync application files before application startup. There could be slower persistent media like NFS and startup takes a long time, then you get into trouble with the readinessProbe (i know, I can set a startupProbe - but it feels dirty)...

instanceid?

The running application writes an instanceid to the configuration which has to be persisted, otherwise the application will try to re-install after pod restarts.

Can someone explain to me, why this instanceid is needed and why it has to be written into the configuration? I would expect it in the database, but not in the file system. The containerized application should not get any state (we have a database for that) and if it has to, it should be stored in the data directory. Otherwise scaling the application is a pain as well.

I would be really happy if someone could shed some light at this rsync idea. Until then, I'll have to build my own image.

Thank you very much :)

BR Matthias

Related issues: https://github.com/nextcloud/docker/issues/1006 https://github.com/nextcloud/docker/issues/1582 https://github.com/nextcloud/docker/issues/1050 https://github.com/nextcloud/docker/issues/869

[J0WI]

The /var/www/html volume is defined in the base image and cannot be undefined. Nextcloud itself does not really work on read-only systems. The config is only one example, apps and themes are also just loaded into the source folder. That's unfortunately very common for PHP applications but consistent with many other PHP Docker images.

The rsync script makes it possible to update the sources when you update the Docker image to a new tag. Otherwise you would have to rely on the internal Nextcloud updater that changes your sources regardless of the Docker image tag.

[jpuskar]

While we're at it -- a full nextcloud k8s operator would be amazing ;).

[J0WI]

You might be also interested in https://github.com/nextcloud/helm/

[chrisamti]

While we're at it -- a full nextcloud k8s operator would be amazing ;).

yeah - but only if the nextcloud docker image removes the rsync stuff and tries to embrace the proper way how a docker image should be done.

And btw. We also need to build our own images because of the rsync stuff...

[chrisamti]

The /var/www/html volume is defined in the base image and cannot be undefined. Nextcloud itself does not really work on read-only systems. The config is only one example, apps and themes are also just loaded into the source folder. That's unfortunately very common for PHP applications but consistent with many other PHP Docker images.

The rsync script makes it possible to update the sources when you update the Docker image to a new tag. Otherwise you would have to rely on the internal Nextcloud updater that changes your sources regardless of the Docker image tag.

if you check how the owncloud people did the docker image, you will see that it's not so difficult to create a proper docker image without the need of rsync.

So your thesis "Nextcloud itself does not really work on read-only systems" is simply wrong.

[J0WI]

The Owncloud images are build completely different and are not part of the official library anymore. AFAIK they're also moving away from PHP.

With this image you can add additional files, configs, hook scripts or volumes. You can also just overwrite the entrypoint of this image to skip rsync completely. If you need to customize it even further, its totally fine to build your own image. This image might be used as a base for all the required runtime libraries.

[tamcore]

AFAIK they're also moving away from PHP.

ownCloud itself will always be PHP. ownCloud Infinite Scale is a complete rewrite in Go.

[WhiteBahamut]

Just want to understand this one better, as the rsync also is annoying in my setup (it takes pretty long)

The /var/www/html volume is defined in the base image and cannot be undefined.

So what you are saying is, the path is defined and people us it. Changing the volume definition would be a breaking change. Is that correct?

From a technical perspective I don't see a reason why the image can't drop the volume definition, copies the Nextcloud files to /var/www/html as part of the image build. As a user I still can mount my persistent folders like custom_apps, themes, config, … into the correct subfolder. The docs for the image even specify this case.

volumeMounts:
        - name: volume-name
          mountPath: /var/www/html/custom_apps
          subPath: customapps
        - name: volume-name
          mountPath: /var/www/html/config
          subPath: config
        - name: volume-name
          mountPath: /var/www/html/data
          subPath: data

I am wondering if we could introduce an env variable to solve this. DO_RSYNC (better name required) defaults to true. We keep the existing use case with rsync and volume of /var/www/html. But if set to false we do not rsync and can mount fine-grained. And we drop the unnamed volume for /var/www/html definition in the Dockerfile Draw back looks to me to have the Nextcloud file twice in the image. But I would swallow that pill (have not checked the size tho)

[chrisamti]

our solution on building customised one volume nextcloud docker images using symlinks, can be found here:

https://gitlab.eqipe.ch/k8s/nextcloud-image/-/blob/main/Dockerfile-apache-one-data-volume.template?ref_type=heads

We use several tricks to overcome the design flaws of the current nextcloud docker image.

Besides installing some more tools into the docker image, we do the following:

• create a new /custom.upgrade.exclude file containing all file and directory names sited on /usr/src/nextcloud, rm /var/www/html and move /usr/src/nextcloud to /var/www/html (where it belongs) (line 23 to 25)
• remove /usr/src/nextcloud and use /upgrade.exclude list to move back only non static files to /usr/src/nextcloud
• create a new symlink friendly /upgrade.exclude (excluding all now static nextcloud src files and folders)
• replace rsync flags -rlDog by -rlKDog to keep symlinks on rsync (line 36)
• copy custom entrypoint shell shell scripts doing the symlink creation (line 43)
• reduce the image size by starting over from scratch (line 49)
• using python code to extract env vars from original nextcloud docker image and replace $more_env with with it

The rest of the magic is done in https://gitlab.eqipe.ch/k8s/nextcloud-image/-/blob/main/eqipe/apache-symlinked/k8s_init.sh?ref_type=heads

It's using /upgrade.exclude.org to create the symlinks needed by nextcloud and restoring version.php and must be used instead of the default /entrypoint.sh

checkout docker-compose.yml to understand how to bring up this symlinked nc docker image.

I hope, sharing our knowledge, would give some ideas to @J0WI how to improve the way nextcloud docker image is done.

All the rearrange steps described above wouldn't be needed of the official docker images created in a more clever way and hopefully in the feature we get a better nc docker image some day...

[thefirstofthe300]

I'm not sure I buy the whole "php doesn't play well with immutable file systems." How does the Nextcloud snap perform upgrades? I'm looking through their code base right now and I can't find anywhere that they copy the Nextcloud code onto a mutable file-system except in their data export script. I'm thinking there should be a way to accomplish building a Docker container that doesn't perform a copy before exporting.

If I didn't have other things to do in my spare time, I'd take a crack at building this container myself.

[thefirstofthe300]

After a bit of toying around, I managed to get a stateless container working, including a very rudimentary upgrade process which handles database upgrades in-container: https://github.com/thefirstofthe300/docker/tree/master/28/apache-stateless

It breaks several of the assumptions made by the existing containers (namely mounts have to be placed on the config, data, custom_apps, and themes directories), but for those who want to adopt a proper immutable container paradigm, this basic container appears to work.

[GuyPaddock]

You may also want to reference some stuff from https://github.com/Inveniem/nextcloud-azure-aks

[ja573]

After a bit of toying around, I managed to get a stateless container working, including a very rudimentary upgrade process which handles database upgrades in-container: https://github.com/thefirstofthe300/docker/tree/master/28/apache-stateless

It breaks several of the assumptions made by the existing containers (namely mounts have to be placed on the config, data, custom_apps, and themes directories), but for those who want to adopt a proper immutable container paradigm, this basic container appears to work.

Any chance to get a PR into the official one?

[bonjour-py]

here may be a sample, https://github.com/hoellen/docker-nextcloud/

[danieljkemp]

I ended up getting around this by building a new container with the source mounted in /var/www/html instead of an incorrect location

FROM nextcloud:29.0.2-fpm

RUN mv /usr/src/nextcloud/* /usr/src/nextcloud/.* /var/www/html

ENTRYPOINT []
CMD ["php-fpm"]

Unfortunately since /var/www/html is declared as a volume in the dockerfile, it can't be linked, and this does increase the image size but that is preferable to maintaining an entirely new image. I manage upgrades separately so I don't need to use the entrypoint script.

I have it paired with an nginx container:

FROM nextcloud:29.0.2-fpm as source

FROM nginx/nginx-unprivileged:1.27-bookworm

USER root
RUN mkdir -p /var/www/html
USER nginx

COPY --from=source /usr/src/nextcloud/ /var/www/html/

This is far preferable to rsyncing the application source between into a shared volume which is the only reason someone might want the entrypoint script doing that.