CollaboraOnline / online

Collabora Online is a collaborative online office suite based on LibreOffice technology. This is also the source for the Collabora Office apps for iOS and Android.
https://collaboraonline.com
Other
1.84k stars 701 forks source link

Collabora spawns hundreds of containers and eats up storage, when executing `occ preview:generate-all` in Nextcloud #10289

Open ErikSteiner opened 1 week ago

ErikSteiner commented 1 week ago

Describe the Bug

I have set up both Nextcloud and Collabora as Docker containers. Nextcloud and Collabora are each their own Docker stack, which can communicate with each other via a Docker network. Nextcloud and Collabora each have their own FQDN.

To increase the performance of Nextcloud, the Preview Generator was installed and a cron job was set up on the Docker host. As required by the documentation.

Cron looks like this:

*/10 * * * * * docker compose -f /docker/nextcloud/docker-compose.yml exec app php occ preview:generate-all

Within a few minutes after the first execution of preview:generate-all, the Docker host is no longer accessible because the entire hard disc space of root has been used up. Until the Docker host was switched off, Collabora's volume totaled 19 GB, which apparently consisted of hundreds of 400 MB containers.

The error occurred when Nextcloud was started. Previously, hundreds of files consisting of documents, images, etc. were manually moved to the user directory of a Nextcloud user while the Nextcloud was offline. When Nextcloud was switched on, the cron job preview:generate-all was used to generate previews for these files. In this respect, the intended result. With the exception that, in my opinion, the mass creation of containers in Collabora should not be intended in this way.

For completeness, excerpts from the config.php, for which files previews are to be created:

  'enabledPreviewProviders' =>
  array (
    0 => 'OC\\Preview\\MP3',
    1 => 'OC\\Preview\\TXT',
    2 => 'OC\\Preview\\MarkDown',
    3 => 'OC\\Preview\\OpenDocument',
    4 => 'OC\\Preview\\Krita',
    5 => 'OC\\Preview\\Image',
    6 => 'OC\\Preview\\HEIC',
    7 => 'OC\\Preview\\TIFF',
  ),
  'preview_imaginary_url' => 'http://imaginary:9000',
  'preview_concurrency_all' => '6',
  'preview_concurrency_new' => '4',

The error seems to have been occurring for some time, as I was able to find it from the following old repository.

Steps to Reproduce

The error should probably be reproducible like this:

  1. set up Nextcloud via Docker
  2. install preview generator and set up cron job
  3. install and set up Collabora, then connect to Nextcloud
  4. store numerous documents on the server via synchronization or manually so that the generator is triggered.

If you need more information about the Docker stacks, I will be happy to help.

Expected Behavior

It should be possible for Collabora to limit the maximum number of open documents. At least I understand the error in this regard. My guess is that every document is opened in Collabora for which the preview generator wants to create a preview. This leads to an overall overload of the service. I cannot judge whether the containers are closed after some time and thus deleted, which should actually save memory space.

Actual Behavior

The preview generator leads to the unlimited generation of Collabora containers and thus to the consumption of the entire hard disc memory.

Desktop

(Please complete the following information)

Additional Context

Original issue: https://github.com/CollaboraOnline/Docker-CODE/issues/79

jazevedo-coll commented 1 week ago

This might address the issue: https://github.com/CollaboraOnline/online/pull/10231