search

nextcloud / activity

:zap: Activity app for Nextcloud
113 stars 64 forks source link

Failed sending activity email messages for deleted (ldap) users spams log #997

Open jpkop opened 1 year ago

jpkop commented 1 year ago

Hello.

we face the following issue: Some former users were deleted via ldap. It seems that there were pending activity mails for these users in the outbox that can not be delivered. Each time the cron job runs we get the log entries down below. We first saw this problems after upgrading to 25.0.1 (from 24.0.5).

Is there a way to manually purge this outbox or solve the issue otherwise?

Thanks in advance.

Server configuration

Operating system: Ubuntu 20.04.5

Web server: nginx

Database: postgresql

PHP version: 8.1.12

Nextcloud version: 25.0.1.1

Where did you install Nextcloud from: docker-compose file

version: '3'
services:
  db:
    image: postgres:13
    restart: always
    logging:
        driver: "json-file"
        options:
            max-file: "10"
            max-size: "10m"
    volumes:
      - ./db:/var/lib/postgresql/data
    env_file:
      - db.env

  redis:
    image: redis
    command: redis-server --requirepass ***
    logging:
        driver: "json-file"
        options:
            max-file: "10"
            max-size: "10m"
    restart: always

  app:
    container_name: app-server
    image: nextcloud:25.0.1-fpm
    restart: always
    logging:
        driver: "json-file"
        options:
            max-file: "10"
            max-size: "10m"
    expose:
      - '80'
      - '9000'
    volumes:
      - ./nextcloud:/var/www/html
      - ./www.overloaded.conf:/usr/local/etc/php-fpm.d/www.overloaded.conf
      - ./nextcloud.ini:/usr/local/etc/php/conf.d/nextcloud.ini
    environment:
      - POSTGRES_HOST=db
      - REDIS_HOST=redis
      - REDIS_HOST_PASSWORD=***
    env_file:
      - db.env
    depends_on:
      - db
      - redis
  cron:
    image: nextcloud:25.0.1-fpm
    restart: always
    logging:
        driver: "json-file"
        options:
            max-file: "10"
            max-size: "10m"
    volumes:
      - ./nextcloud:/var/www/html
    entrypoint: /cron.sh
    depends_on:
      - db
      - redis
      - app

  onlyoffice-document-server:
    container_name: onlyoffice-document-server
    image: onlyoffice/documentserver:7.1
    restart: always
    logging:
        driver: "json-file"
        options:
            max-file: "10"
            max-size: "10m"
    expose:
      - '80'
      - '443'
    volumes:
      - ./document_data:/var/www/onlyoffice/Data
      - ./document_log:/var/log/onlyoffice
  nginx:
    container_name: nginx-server
    image: nginx:1.21
    restart: always
    logging:
        driver: "json-file"
        options:
            max-file: "10"
            max-size: "10m"
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./cloud-cert.pem:/etc/nginx/cloud-cert.pem
      - ./private.pem:/etc/nginx/private.pem
      - ./nextcloud:/var/www/html
    depends_on:
      - db
      - redis
      - app

volumes:
  db:
  nextcloud:
  document_data:
  document_log:
  app_data:

Signing status: Not sure what this means

Nextcloud configuration:

{
    "system": {
        "minimum.supported.desktop.version": "3.4.1",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***",
            "nginx-server"
        ],
        "default_language": "de",
        "default_locale": "de",
        "default_phone_region": "de",
        "knowledgebaseenabled": true,
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "25.0.1.1",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "overwritehost": "***REMOVED SENSITIVE VALUE***",
        "overwriteprotocol": "https",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "onlyoffice": {
            "DocumentServerUrl": "\/ds-vpath\/",
            "DocumentServerInternalUrl": "http:\/\/onlyoffice-document-server\/",
            "StorageUrl": "http:\/\/nginx-server\/",
            "verify_peer_off": true
        },
        "allow_local_remote_servers": "true",
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "maintenance": false,
        "maintenance_window_start": 1,
        "mail_smtpauthtype": "PLAIN",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "tls",
        "auto_logout": false,
        "loglevel": 2,
        "app_install_overwrite": [
            "spreed",
            "files_texteditor",
            "tasks",
            "quota_warning",
            "forms",
            "files_mindmap",
            "files_bpm",
            "drawio"
        ]
    }
}

Logs

Nextcloud log (data/nextcloud.log)

[activity] Warnung: Failed sending activity email to user '3ef6d5ba-f34d-103a-9a21-73d5657b9a5b'. at 2022-11-08T13:10:02+00:00

[activity] Fehler: Swift_TransportException: Expected response code 354 but got code "554", with message "554 5.5.1 Error: no valid recipients" at <<closure>>

 0. /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php line 344
    Swift_Transport_AbstractSmtpTransport->assertResponseCode("554 5.5.1 Error ... n", [354])
 1. /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/EsmtpTransport.php line 305
    Swift_Transport_AbstractSmtpTransport->executeCommand("DATA\r\n", [354], ["user@user@domain.tld"], false, null)
 2. /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php line 392
    Swift_Transport_EsmtpTransport->executeCommand("DATA\r\n", [354], ["user@user@domain.tld"])
 3. /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php line 499
    Swift_Transport_AbstractSmtpTransport->doDataCommand(["user@domain.tld"])
 4. /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php line 518
    Swift_Transport_AbstractSmtpTransport->doMailTransaction(Swift_Message {}, "cloudmail@domain.tld", ["user@domain.tld"], ["user@domain.tld"])
 5. /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php line 206
    Swift_Transport_AbstractSmtpTransport->sendTo(Swift_Message {}, "cloudmail@domain.tld", ["3ef6d5ba-f34d-103a-9a21-73d5657b9a5b"], ["user@domain.tld"])
 6. /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Mailer.php line 71
    Swift_Transport_AbstractSmtpTransport->send(Swift_Message {}, ["user@domain.tld"])
 7. /var/www/html/lib/private/Mail/Mailer.php line 191
    Swift_Mailer->send(Swift_Message {}, ["user@domain.tld"])
 8. /var/www/html/apps/activity/lib/MailQueueHandler.php line 415
    OC\Mail\Mailer->send(OC\Mail\Message {})
 9. /var/www/html/apps/activity/lib/MailQueueHandler.php line 165
    OCA\Activity\MailQueueHandler->sendEmailToUser("3ef6d5ba-f34d-103a-9a21-73d5657b9a5b", "user@domain.tld", "de", "Europe/Berlin", 1667913000)
10. /var/www/html/apps/activity/lib/BackgroundJob/EmailNotification.php line 61
    OCA\Activity\MailQueueHandler->sendEmails(500, 1667913000)
11. /var/www/html/lib/private/BackgroundJob/Job.php line 54
    OCA\Activity\BackgroundJob\EmailNotification->run(null)
12. /var/www/html/lib/private/BackgroundJob/TimedJob.php line 60
    OC\BackgroundJob\Job->execute(OC\BackgroundJob\JobList {}, OC\Log {})
13. /var/www/html/cron.php line 152
    OC\BackgroundJob\TimedJob->execute(OC\BackgroundJob\JobList {}, OC\Log {})
at 2022-11-08T13:10:02+00:00

[PHP] Fehler: Error: strtolower(): Passing null to parameter #1 ($string) of type string is deprecated at /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/EsmtpTransport.php#144 at <<closure>>

 0. <<closure>>
    OC\Log\ErrorHandler::onError(8192, "strtolower(): P ... d", "/var/www/html/3 ... p", 144)
 1. /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/EsmtpTransport.php line 144
    strtolower(null)
 2. /var/www/html/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/SmtpTransport.php line 43
    Swift_Transport_EsmtpTransport->setEncryption(null)
 3. /var/www/html/lib/private/Mail/Mailer.php line 267
    Swift_SmtpTransport->__construct()
 4. /var/www/html/lib/private/Mail/Mailer.php line 254
    OC\Mail\Mailer->getSmtpInstance()
 5. /var/www/html/lib/private/Mail/Mailer.php line 180
    OC\Mail\Mailer->getInstance()
 6. /var/www/html/apps/activity/lib/MailQueueHandler.php line 415
    OC\Mail\Mailer->send(OC\Mail\Message {})
 7. /var/www/html/apps/activity/lib/MailQueueHandler.php line 165
    OCA\Activity\MailQueueHandler->sendEmailToUser("3ef6d5ba-f34d-103a-9a21-73d5657b9a5b", "user@domain.tld, "de", "Europe/Berlin", 1667913899)
 8. /var/www/html/apps/activity/lib/BackgroundJob/EmailNotification.php line 61
    OCA\Activity\MailQueueHandler->sendEmails(500, 1667913899)
 9. /var/www/html/lib/private/BackgroundJob/Job.php line 54
    OCA\Activity\BackgroundJob\EmailNotification->run(null)
10. /var/www/html/lib/private/BackgroundJob/TimedJob.php line 60
    OC\BackgroundJob\Job->execute(OC\BackgroundJob\JobList {}, OC\Log {})
11. /var/www/html/cron.php line 152
    OC\BackgroundJob\TimedJob->execute(OC\BackgroundJob\JobList {}, OC\Log {})

at 2022-11-08T13:25:01+00:00
nickvergessen commented 1 year ago

The error should also happen when sending the test email from index.php/settings/admin right?

jpkop commented 1 year ago

The error should also happen when sending the test email from index.php/settings/admin right?

The error message below does appear in the log, the upper ones do not (since the mail is adressed to me and not to the deleted users).

silvan78 commented 1 year ago

+1, Problem persists in NextCloud 24.0.8 Users removed from ladp, system still trying to send emails to removed (from ldap group) users.

{"reqId":"968SxTD1FT27CdfgY6bs","level":3,"time":"2023-08-30T12:30:38+00:00","remoteAddr":"","user":"--","app":"activity","method":"","url":"--","message":"Failed sending activity email to user \"bd8ad2dc-0129-103d-901a-6f7a8ddc7eb7\"","userAgent":"--","version":"25.0.8.2","exception":{"Exception":"Swift_TransportException","Message":"Expected response code 354 but got code \"503\", with message \"503-All RCPT commands were rejected with this error:\r\n503-This user shouldn't receive messages\r\n503 Valid RCPT command must precede DATA\r\n\"","Code":503,"Trace":[{"file":"/var/www/html/nextcloud/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php","line":344,"function":"assertResponseCode","class":"Swift_Transport_AbstractSmtpTransport","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/EsmtpTransport.php","line":305,"function":"executeCommand","class":"Swift_Transport_AbstractSmtpTransport","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php","line":392,"function":"executeCommand","class":"Swift_Transport_EsmtpTransport","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php","line":499,"function":"doDataCommand","class":"Swift_Transport_AbstractSmtpTransport","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php","line":518,"function":"doMailTransaction","class":"Swift_Transport_AbstractSmtpTransport","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php","line":206,"function":"sendTo","class":"Swift_Transport_AbstractSmtpTransport","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Mailer.php","line":71,"function":"send","class":"Swift_Transport_AbstractSmtpTransport","type":"->"},{"file":"/var/www/html/nextcloud/lib/private/Mail/Mailer.php","line":191,"function":"send","class":"Swift_Mailer","type":"->"},{"file":"/var/www/html/nextcloud/apps/activity/lib/MailQueueHandler.php","line":415,"function":"send","class":"OC\Mail\Mailer","type":"->"},{"file":"/var/www/html/nextcloud/apps/activity/lib/MailQueueHandler.php","line":165,"function":"sendEmailToUser","class":"OCA\Activity\MailQueueHandler","type":"->"},{"file":"/var/www/html/nextcloud/apps/activity/lib/BackgroundJob/EmailNotification.php","line":61,"function":"sendEmails","class":"OCA\Activity\MailQueueHandler","type":"->"},{"file":"/var/www/html/nextcloud/lib/private/BackgroundJob/Job.php","line":54,"function":"run","class":"OCA\Activity\BackgroundJob\EmailNotification","type":"->"},{"file":"/var/www/html/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":60,"function":"execute","class":"OC\BackgroundJob\Job","type":"->"},{"file":"/var/www/html/nextcloud/cron.php","line":152,"function":"execute","class":"OC\BackgroundJob\TimedJob","type":"->"}],"File":"/var/www/html/nextcloud/3rdparty/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php","Line":459,"message":"Failed sending activity email to user \"{user}\"","CustomMessage":"Failed sending activity email to user \"bd8ad2dc-0129-103d-901a-6f7a8ddc7eb7\""},"id":"64ef366edf63e"}

silvan78 commented 1 year ago

I looked further into subject and Activity might be only partially wrong. The users that generate the error are no longer in occ user:list, but then i ask occ:info {cloud_id} i stil get data:

sudo -u www-data php /var/www/html/nextcloud/occ user:info bd8ad2dc-0129-103d-901a-6f7a8ddc7eb7
  - user_id: bd8ad2dc-0129-103d-901a-6f7a8ddc7eb7
  - display_name: bd8ad2dc-0129-103d-901a-6f7a8ddc7eb7
  - email: #REDACTED#
  - cloud_id: bd8ad2dc-0129-103d-901a-6f7a8ddc7eb7@#REDACTED#
  - enabled: true
  - groups:
  - quota: 10 GB
  - storage:
  - last_seen: 1970-01-01T00:00:00+00:00
  - user_directory: #REDACTED#
  - backend: LDAP

Thus, I would suspect, that Activity tries to send activity emails to all detected users, that are still in the system (even is not visible). The nextcloud does not delete these accounts (which might be desired behaviour).

The desired behaviour would be to take list for processing from occ user:list like source, not DB itself.

edit. I looked further, Nextcloud has a background process called LDAP user cleanup that marks disconnected accounts. They can be seen with: sudo -u www-data php /var/www/html/nextcloud/occ ldap:show-remnants

The idea is to filter these entries from Activity email processing list.

nickvergessen commented 1 year ago

Problem persists in NextCloud 24.0.8

That version is not supported anymore, please update to a supported version and try again.

m3e-g commented 8 months ago

I had the same issue with the disabled LDAP users, and It's been going on for awhile now. Most recent version on my side was 28.0.1.1 and the issue persists.

ThibautPlg commented 5 months ago

Hi, From my understanding, emails should not be sent to disabled users (see this condition ). However, from my experience, remnants LDAP users have a isDeleted configkey set to 1 in oc_preferences but that doesn't disable them. (as seen through a occ user:info on the same userid)

Activity app might not be the right place to address this problem as this app isn't supposed to cover all user backends possible. Maybe a user_ldap issue?

I'm also concerned by the issue of Nextcloud trying to send email notifications to remnants. @nickvergessen What's your take on this?

Best regards,

ThibautPlg commented 3 months ago

Hello Any news on this subject? Regards,