[Bug]: NC doesn't send an email reply to accepted meetings using thunderbird

section1 commented 1 year ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] Nextcloud Server is running on 64bit capable CPU, PHP and OS.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

I have NC version 25.0.5 with calendar 4.3.4. Using Thunderbird as caldav client of NC. When i receive an invitation from gmail/Outlook accounts and i accept the meeting using Thunderbird NC creates the event in my calendar but never send a Reply saying that i accepted/declined the invitation.

If i send an invitation from other NC user using the calendar app and i accept from Thunderbird i get a reply.

The pŕoblem looks like when the organizer is outside the calendar, let's say an external invitation.

Steps to reproduce

Create and invite in gmail calendar using as 'Guest' your email associated to NC user.
In Thunderbird Accept or Decline the meeting.
Check the status of the event in the Calendar web page.
NC never sends a Reply

Expected behavior

NC sends a reply accepting or declining the invitation.

Installation method

Community Manual installation with Archive

Nextcloud Server version

25

Operating system

Debian/Ubuntu

PHP engine version

PHP 7.4

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

[ ] Default user-backend (database)
[X] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "skeletondirectory": "",
        "defaultapp": "calendar",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "calendar.domain.tld"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "25.0.5.1",
        "overwrite.cli.url": "https:\/\/calendar.domain.tld",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "maintenance": false,
        "session_lifetime": 14400,
        "remember_login_cookie_lifetime": 172800
    }
}

List of activated Apps

Enabled:
  - calendar: 4.3.4
  - circles: 25.0.0
  - cloud_federation_api: 1.8.0
  - contactsinteraction: 1.6.0
  - dav: 1.24.0
  - federatedfilesharing: 1.15.0
  - federation: 1.15.0
  - files: 1.20.1
  - files_rightclick: 1.4.0
  - files_trashbin: 1.15.0
  - files_versions: 1.18.0
  - logreader: 2.10.0
  - lookup_server_connector: 1.13.0
  - nextcloud_announcements: 1.14.0
  - notifications: 2.13.1
  - oauth2: 1.13.0
  - password_policy: 1.15.0
  - privacy: 1.9.0
  - provisioning_api: 1.15.0
  - related_resources: 1.0.4
  - serverinfo: 1.15.0
  - settings: 1.7.0
  - systemtags: 1.15.0
  - theming: 2.0.1
  - twofactor_backupcodes: 1.14.0
  - updatenotification: 1.15.0
  - user_ldap: 1.15.0
  - user_status: 1.5.0
  - viewer: 1.9.0
  - workflowengine: 2.7.0
Disabled:
  - activity: 2.17.0
  - admin_audit
  - bruteforcesettings
  - comments: 1.15.0
  - dashboard: 7.5.0
  - encryption
  - files_external
  - files_pdfviewer: 2.6.0
  - files_sharing: 1.17.0
  - firstrunwizard: 2.14.0
  - photos: 2.0.1
  - recommendations: 1.4.0
  - sharebymail: 1.15.0
  - support: 1.8.0
  - survey_client: 1.13.0
  - suspicious_login
  - text: 3.6.0
  - twofactor_totp
  - weather_status: 1.5.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

No response

flortsch commented 7 months ago

Same scenario, same issue here as well. With both Thunderbird 115.8.1 and Evolution 3.50.4. On Nextcloud 27.1.7. Prefer client-side scheduling is disabled on both clients. The NC calendar is a personal one and not a shared one.

flortsch commented 7 months ago

Same on Nextcloud 28.0.3.

flortsch commented 7 months ago

I enabled xdebug in our Nextcloud instance and debugged the code that handles the creation of new calendar events. Here are my insights.

It looks to me as if this case was never really handled in the code. There are two event handlers that are triggered when Thunderbird or another client creates a calendar event after accepting the external invitation: beforeCreateFile and afterCreateFile.

The beforeCreateFile handler calls into the parseEvent method of the 3rdparty sabre-vobject ITip\Broker class. Here, the following code for some reason I don't know skips calling parseEventForAttendee when a new event is created (which would return a new reply message I guess). $oldCalendar is null when a new event is created. Only event updates are handled in the code. Should this code handle the event creation case as well? Here is the code:

    public function parseEvent($calendar, $userHref, $oldCalendar = null)
    {
        // ...

        if (in_array($eventInfo['organizer'], $userHref)) {
            return $this->parseEventForOrganizer($baseCalendar, $eventInfo, $oldEventInfo);
        } elseif ($oldCalendar) {
            // We need to figure out if the user is an attendee, but we're only
            // doing so if there's an oldCalendar, because we only want to
            // process updates, not creation of new events.
            foreach ($eventInfo['attendees'] as $attendee) {
                if (in_array($attendee['href'], $userHref)) {
                    return $this->parseEventForAttendee($baseCalendar, $eventInfo, $oldEventInfo, $attendee['href']);
                }
            }
        }

        return [];
    }

The afterCreateFile handler calls into the dispatchSchedulingResponses method of Nextclouds CalDAV\Schedule\Plugin class. I don't understand how this code exactly works, because the method skips calls to scheduleLocalDelivery as $this->schedulingResponses is empty but is set in scheduleLocalDelivery itself. This tells me that this code path must be handling some different cases.

        public function dispatchSchedulingResponses(string $uri):void {
        if ($uri !== $this->pathOfCalendarObjectChange) {
            return;
        }

        foreach ($this->schedulingResponses as $schedulingResponse) {
            $this->scheduleLocalDelivery($schedulingResponse);
        }
    }

Maybe a groupware member (@ChristophWurst) can have a deeper look into this and tell what would be the best way to fix/implement this.

ChristophWurst commented 7 months ago

Could this be the same problem as https://github.com/nextcloud/server/issues/43115?

flortsch commented 7 months ago

I will dig deeper and see if the issue is related in the next days. After a quick view I don't see how the schedule method of the IMipPlugin comes into play after event creation, but I am not familiar with the code base. Is there some kind of background job or other entrypoint that triggers the schedule method after event creation?

ChristophWurst commented 7 months ago

AFAIK it is triggered through an event within the same request

flortsch commented 7 months ago

What I currently see is this: parseEvent in above mentioned code snippet returns an empty messages array as a result to the callee processICalendarChange and therefore, no messages are delivered, i.e., no call to deliver or schedule happens. schedule would call the buggy method getCurrentAttendee referenced in your mentioned issue. If there is nothing else that triggers notifications for event creations then I think the issue is unrelated.

flortsch commented 3 months ago

I think this will be fixed by https://github.com/nextcloud/server/pull/44893 as well.

section1 commented 3 months ago

Hi @flortsch today i tested that patch in an NC version 27.1.11(NOT 28.x) and still have this issue from external users(not NC users).

section1 commented 3 months ago

Too I did this test...i sent an event from gmail y download the ics from thunderbird and then i go to the calendar app in NC and import the ics from there. Then i refresh my calendar in thunderbird and if i accept that event the email notification works fine. (This same event accepted via thunderbird the email notifications fails)

This maybe is a problem with thunderbird ?? not sure, but some time ago i opened this case to mozilla.

flortsch commented 3 months ago

Hello @section1, sorry for the confusion. I was too optimistic and you are right, https://github.com/nextcloud/server/pull/44893 does not fix this issue. Tested the patch as well with Evolution. Nextcloud still does not seem to trigger email replies when an external client like Thunderbird or Evolution is used with client-side scheduling disabled.

But good news is that with this patch, Nextcloud now at least triggers email replies when one uses Nextcloud calendar or Nextcloud mail to accept invites or to update the event.

SebastianKrupinski commented 3 months ago

Fixed with #44893

flortsch commented 3 months ago

Fixed with #44893

Please see the comments above. This is not fixed by #44893. Nextcloud still does not send out email replies when external clients like Thunderbird or Evolution are used with client-side scheduling disabled in the client.

SailReal commented 3 months ago

@SebastianKrupinski please reopen this issue as this bug has not been fixed by https://github.com/nextcloud/server/pull/44893 ... could create a new issue, but that would be even more confusing imo.

SebastianKrupinski commented 3 months ago

Hi @flortsch and @SailReal

Okay, I have reopened this... I can have another look in to it...

SebastianKrupinski commented 3 months ago

@SailReal @flortsch

So I am trying to understand the issue... is the issue that when you get an external invite... There is NO reply message generate when the event is saved in your Calendar?

SailReal commented 3 months ago

So I am trying to understand the issue... is the issue that when you get an external invite... There is NO reply message generate when the event is saved in your Calendar?

Exactly :+1: and can be reproduced using the "Steps to reproduce" section of https://github.com/nextcloud/server/issues/38651#issue-1742220422

nextcloud / server