nextcloud / previewgenerator

Nextcloud app to do preview generation in the background.
https://apps.nextcloud.com/apps/previewgenerator
GNU Affero General Public License v3.0
450 stars 56 forks source link

soffice.bin <defunct> and many soffice.bin processes for xlsx, docx, pptx files #302

Closed francoisPE closed 3 months ago

francoisPE commented 2 years ago

Hello,

I face my nextcloud killing my server due to dozen of child processes sent suddenly over max_children allowed. Increasing memory_limit helps, obviously higher max_children helps also. Unfortunately, even upgraded my VPS still has difficulties to manage load.

Digging, I saw that chilren generation match preview operations and specifically xlsx, docx, pptx ps aux | grep -c /soffice.bin gives 42 processes running for several days ps aux | egrep -c "Z|defunct" gives 41 defunct processes...

In my setup, I try remove OpenDocument class, occ config:system:delete enabledPreviewProviders 1 --value='OC\Preview\OpenDocument' but, nothing changed

Here my config.php

  'enabledPreviewProviders' =>
  array (
    1 => 'OC\\Preview\\PNG',
    2 => 'OC\\Preview\\HEIC',
    3 => 'OC\\Preview\\GIF',
    4 => 'OC\\Preview\\TIFF',
    5 => 'OC\\Preview\\BMP',
    6 => 'OC\\Preview\\XBitmap',
    7 => 'OC\\Preview\\MP3',
    8 => 'OC\\Preview\\TXT',
    9 => 'OC\\Preview\\MarkDown',
    10 => 'OC\\Preview\\Krita',
    11 => 'OC\\Preview\\MSOffice2003',
    12 => 'OC\\Preview\\MSOffice2007',
    13 => 'OC\\Preview\\MSOfficeDoc',
    14 => 'OC\\Preview\\PDF',
    15 => 'OC\\Preview\\SVG',
    16 => 'OC\\Preview\\Movie',
    17 => 'OC\\Preview\\Font',
    18 => 'OC\\Preview\\MKV',
    19 => 'OC\\Preview\\MP4',
    20 => 'OC\\Preview\\AVI',
    21 => 'OC\\Preview\\Photoshop',
    22 => 'OC\\Preview\\CSV',
  ),

Moreover trying to run occ preview:generate-all, job stayed stuked in a folder for hours... In log, I found nothing except acces 404 or 504 for some previews.

I run every 5 minutes a cron job php -d memory_limit=1024M occ preview:pre-generate cron log gives nothing. but related processes used up to 80% CPU...

My server ubuntu 20.04 apache 2.4 mariadb Ver 15.1 Distrib 10.3.34-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

Nextcloud 23.0.5 preview generator 5.0.0

my config.php

$CONFIG = array (
  'passwordsalt' => '******',
  'secret' => '***********',
  'trusted_domains' =>
  array (
    0 => 'localhost',
    1 => 'nc.********.com',
  ),
  'datadirectory' => '/mypath',
  'dbtype' => 'mysql',
  'version' => '23.0.5.1',
  'overwrite.cli.url' => 'https://nc.*********.com/',
  'dbname' => 'c**********_db',
  'dbhost' => 'localhost',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'mysql.utf8mb4' => true,
  'dbuser' => 'c***********',
  'dbpassword' => '*',
  'installed' => true,
  'instanceid' => 'o*****',
  'default_language' => 'fr',
  'default_locale' => 'fr',
  'default_phone_region' => 'fr',
  'defaultapp' => 'dashboard,files,mail,contacts,calendar,notes,deck,tasks,spreed,passwords,photos,fulltextsearch,duplicatefinder,activity',
  'knowledgebaseenabled' => true,
  'remember_login_cookie_lifetime' => 1296000,
  'session_lifetime' => 86400,
  'session_keepalive' => true,
  'filesystem_check_changes' => 0,
  'mail_from_address' => 'contact',
  'mail_smtpmode' => 'smtp',
  'mail_sendmailmode' => 'smtp',
  'mail_domain' => '***********',
  'mail_smtpsecure' => 'tls',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtpauth' => 1,

  'mail_smtphost' => '************',
  'mail_smtpport' => '587',
  'mail_smtpname' => 'contact@*********',
  'mail_smtppassword' => '**********',
  'htaccess.RewriteBase' => '/',
  'trashbin_retention_obligation' => 'auto, 7',
  'versions_retention_obligation' => 'auto, 7',
  'log_type' => 'file',
  'logfile' => '/mypath/log.log',
  'log_rotate_size' => 20971520,
  'logfilemode' => 640,
  'loglevel' => 3,
  'syslog_tag' => '*********',
  'logtimezone' => 'Europe/Paris',
  'redis' =>
  array (
    'host' => '127.0.0.1',
    'port' => 6379,
    'password' => '******************',
    'dbindex' => '2',
  ),
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'memcache.local' => '\\OC\\Memcache\\APCu',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'upgrade.disable-web' => true,
  'ldapProviderFactory' => 'OCA\\User_LDAP\\LDAPProviderFactory',
  'enable_previews' => true,
  'preview_max_x' => '512',
  'preview_max_y' => '512',
  'preview_max_filesize_image' => '5',
  'preview_libreoffice_path' => '/usr/bin/libreoffice',
  'enabledPreviewProviders' =>
  array (
    1 => 'OC\\Preview\\PNG',
    2 => 'OC\\Preview\\HEIC',
    3 => 'OC\\Preview\\GIF',
    4 => 'OC\\Preview\\TIFF',
    5 => 'OC\\Preview\\BMP',
    6 => 'OC\\Preview\\XBitmap',
    7 => 'OC\\Preview\\MP3',
    8 => 'OC\\Preview\\TXT',
    9 => 'OC\\Preview\\MarkDown',
    10 => 'OC\\Preview\\Krita',
    11 => 'OC\\Preview\\MSOffice2003',
    12 => 'OC\\Preview\\MSOffice2007',
    13 => 'OC\\Preview\\MSOfficeDoc',
    14 => 'OC\\Preview\\PDF',
    15 => 'OC\\Preview\\SVG',
    16 => 'OC\\Preview\\Movie',
    17 => 'OC\\Preview\\Font',
    18 => 'OC\\Preview\\MKV',
    19 => 'OC\\Preview\\MP4',
    20 => 'OC\\Preview\\AVI',
    21 => 'OC\\Preview\\Photoshop',
    22 => 'OC\\Preview\\CSV',
  ),
  'log.condition' => '[\'apps\' => [\'admin_audit\'],]',
  'activity_expire_days' => 15,
  'appstoreenabled' => false,
  'maintenance' => false,
  'skeletondirectory' => '/mypath/skeleton',
  'templatedirectory' => '/mypath/skeleton/Modèles',
  'theme' => '',
);

and

        "previewgenerator": {
            "enabled": "yes",
            "heightSizes": "34 128 512",
            "installed_version": "5.0.0",
            "squareSizes": "32 64 512",
            "types": "filesystem",
            "widthSizes": "32 128 512"
        },
joshtrichards commented 3 months ago
    11 => 'OC\\Preview\\MSOffice2003',
    12 => 'OC\\Preview\\MSOffice2007',
    13 => 'OC\\Preview\\MSOfficeDoc',

Each of these document types that you still have enabled also utilizes the libreoffice binary (same as soffice.bin) too:

https://github.com/nextcloud/server/blob/50309fe436e156b436323b36fe2e7bbd329c316a/lib/private/PreviewManager.php#L417-L422