[Bug]: Cron runs forever

tograss commented 8 months ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

Since the update to 28 the cron jobs get stuck once a day. I've seen https://github.com/nextcloud/server/issues/42793 and https://github.com/nextcloud/server/issues/43184 but I'm unsure if its the same problem. Feel free to close or link if its the same.

In my case the cronjobs getting not OOM killed instead they keep stuck till so many processes are spanned the system gets slow. When I do a strace I get

lstat("/srv/www/XXX/data/8ED118F1-XXX", 0x7fffc8330550) = -1 ENOENT (Datei oder Verzeichnis nicht gefunden) forever. When I create the missing folder the cronjobs end. However next time they got stuck with a different folder.

Steps to reproduce

update from 27 to 28

Expected behavior

Cronjobs should end

Installation method

None

Nextcloud Server version

28

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.1

Web server

Apache (supported)

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[ ] Default user-backend (database)
[X] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "28.0.1.1",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "forcessl": true,
        "ldapIgnoreNamingRules": false,
        "theme": "",
        "default_language": "en",
        "maintenance": false,
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "secret": "***REMOVED SENSITIVE VALUE***",
        "loglevel": 2,
        "trashbin_retention_obligation": "auto",
        "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "sendmail",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "stable",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_smtpauthtype": "PLAIN",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "app_install_overwrite": [
            "calendar",
            "deck",
            "bruteforcesettings"
        ],
        "encryption.legacy_format_support": false,
        "encryption.key_storage_migrated": false,
        "default_phone_region": "DE",
        "mysql.utf8mb4": true
    }
}

List of activated Apps

Enabled:
  - activity: 2.20.0
  - admin_audit: 1.18.0
  - bruteforcesettings: 2.8.0
  - calendar: 4.6.4
  - circles: 28.0.0-dev
  - cloud_federation_api: 1.11.0
  - comments: 1.18.0
  - contacts: 5.5.1
  - contactsinteraction: 1.9.0
  - dav: 1.29.1
  - deck: 1.12.1
  - federatedfilesharing: 1.18.0
  - federation: 1.18.0
  - files: 2.0.0
  - files_automatedtagging: 1.18.0
  - files_external: 1.20.0
  - files_pdfviewer: 2.9.0
  - files_reminders: 1.1.0
  - files_sharing: 1.20.0
  - files_trashbin: 1.18.0
  - files_versions: 1.21.0
  - firstrunwizard: 2.17.0
  - impersonate: 1.15.0
  - logreader: 2.13.0
  - lookup_server_connector: 1.16.0
  - nextcloud_announcements: 1.17.0
  - notifications: 2.16.0
  - oauth2: 1.16.3
  - password_policy: 1.18.0
  - photos: 2.4.0
  - privacy: 1.12.0
  - provisioning_api: 1.18.0
  - recommendations: 2.0.0
  - related_resources: 1.3.0
  - serverinfo: 1.18.0
  - settings: 1.10.1
  - sharebymail: 1.18.0
  - support: 1.11.0
  - survey_client: 1.16.0
  - systemtags: 1.18.0
  - text: 3.9.1
  - theming: 2.3.0
  - twofactor_backupcodes: 1.17.0
  - updatenotification: 1.18.0
  - user_ldap: 1.19.0
  - user_status: 1.8.1
  - user_usage_report: 1.12.0
  - viewer: 2.2.0
  - workflowengine: 2.10.0
Disabled:
  - dashboard: 7.8.0 (installed 7.0.0)
  - encryption: 2.16.0 (installed 2.3.0)
  - files_rightclick: 0.15.1 (installed 1.6.0)
  - suspicious_login: 6.0.0
  - twofactor_totp: 10.0.0-beta.2
  - weather_status: 1.8.0 (installed 1.0.0)

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

sangfroid70 commented 7 months ago

Same problem for me, up my CPU to 100%. When I try to execute cron.php with the same owner, it finishes normally

ChristophWurst commented 3 months ago

https://github.com/nextcloud/server/pull/45804 will provide a bit more clarity. We will start to see which specific jobs are taking ages to complete.

nextcloud / server