Calendar invites: Administrator shall choose if server's or user's email adress is used

[tuergeist]

Feature Request

I refer to #2345 and #2778.

If a Nextcloud instance serves for users from different domains (which is the case at least for me), it's annoying that calendar invites are sent with the server's email address instead of the user's once. For me it would be fine if the server's address is taken if there is no user email configured.

To avoid problems with mailservers that might not accept sending emails for more than one domain, this behavior this could be made optional. (In my case all domains are hosted and handled by one email server and one nextcloud server - I would not like to set up a nextcloud instance per domain).

(If someone is so nice and point me to the place where I can change it in the code myself, would be fine too)

Implementation

Delegate sending emails to apps. The main target is the Mail app but others may offer this integration too.

Work packages

• [x] Add a mail sending provider mechanism
• [x] Implement the mail sending provider for Mail
• [ ] Add an admin setting to choose which email system should be used (system or mail providers)
• [x] Adjust CalDAV backend to use the providers and system email as fallback
• [x] Developer documentation for the new API
• [ ] Admin documentation for the setup and any configuration options
• [ ] User documentation for any user settings

[georgehrke]

@tuergeist This is the place to go: https://github.com/nextcloud/server/blob/master/apps/dav/lib/CalDAV/Schedule/IMipPlugin.php#L109

Feel free to send a PR that implements this optionally. :)

[georgehrke]

(Message provides a setFrom method, see https://github.com/nextcloud/server/blob/master/lib/private/Mail/Message.php#L81)

[tuergeist]

Hi @georgehrke, IMHO the problem is the sender's address which I cannot change easily in neither code locations you pointed to. [Or I'm just too stupid]

[tuergeist]

Aaahh. Got it.

[robertoschwald]

Any news on this?

[ntnsndr]

I'd add that this is an important issue for us at MayFirst.org, who use the same NextCloud across a great many domains and organizational identities. We really need these invitation emails to be coming from the users who send them. Thanks to all who have the capacity and willingness to work on this.

[skjnldsv]

@georgehrke :trumpet: could you update this? :)

[georgehrke]

cc @jancborchardt This is what we talked about on Talk.

[georgehrke]

to summarise the problem:

• depending on the environment, users will have email-addresses managed by a different organisation than the Nextcloud-instance. If your Nextcloud is configured with the email address noreply@abc.tld, but your users use different providers like gmail, outlook, whatever, your email server (abc.tld) can't just start sending out emails for gmail, outlook, etc. Most of the times, these emails will be marked as spam and your server runs the risk of being put onto blacklists for spamming.

... and our options:

• add an option for admins that allows them to send all emails (or at least emails from the same domain) via the abc.tld email server but with the real user email address. (user1@abc.tld). This should come with a big fat warning that if you enable it despite your users having email addresses from plenty different providers, you might run at the risk of being put onto spam blacklists
• integrate even better with the mail app. Allow the mail app to hook into the IMipPlugin and send out emails if the currently logged in user has an account registered for the email of the Organizer given in the calendar data. If there is no email registered, fallback to the current behaviour. cc @ChristophWurst

[ChristophWurst]

* integrate even better with the mail app. Allow the mail app to hook into the IMipPlugin and send out emails if the currently logged in user has an account registered for the email of the Organizer given in the calendar data. If there is no email registered, fallback to the current behaviour. cc @ChristophWurst

Sounds like a good idea in general.

[jancborchardt]

Sounds good and seems we should do both options?

Everyone who commented in the thread: Would this solve your issue?

[biva]

To be sure to understand the proposal of @georgehrke above:

1. User1, with email address user1@xyz.com will be able to send an invite with his own email address as organizer (and not noreply@abc.tld anymore). This has the risk to be considered as spam. This risk is lower if the email address of User1 is user1@abc.tld, but the risk still exists.
2. If User1 configures the mail app with his setup (his credentials + imap.xyz.com for example), the calendar app will be able to send an invite directly from the mail app.

It sounds good :)

Now, what happens if I use Thunderbird for my emails? Will it also work?

[georgehrke]

I'm super sorry, but all the development power of @ChristophWurst and me are currently bound on other tickets, so this will have to wait for Nextcloud 19.

[gitterdoneplease]

I'm no dev, but you can get this done by editing nextcloudroot/apps/dav/lib/CalDAV/Schedule/IMipPlugin.php

Change line 255 from: ->setFrom([$fromEMail => $fromName]) to: ->setFrom([$sender => $senderName])

And your invites will come from the email address of the user instead of the server.

[ChristophWurst]

@gitterdoneplease this is a bit weird. I'm sure some mail servers will either not allow you to send a message with a mismatching from header or the recipient's mail server will mark it as spam as it looks like someone tries to spoof a sender.

[robertoschwald]

If all users use the same email domain, why should it be blocked?

[georgehrke]

If all users use the same email domain,

This is something we can't assume. See my comment above: https://github.com/nextcloud/server/issues/5080#issuecomment-501158303

[gitterdoneplease]

Dunno gents. I run the mail server too so it’s not an issue for me. A big warning in the GUI should be enough, but this was necessary in my case to transition completely to NC. Spoofing a from header if you AUTH on a SMTP server is painless, but as pointed out may trigger spam/junk detectors.

[Findebug]

So not going to make Nextcloud 19?

[loibpet]

I am aware about quite a few organisations with different email domains. This easily happens with mergers, different legal entities for parts of a company (e.g. regulated business vs. un-regulated business, ...). There are tons of situations where this can happen.

Just don't assume that this is always fixed. Adding flexibility does not hurt at all and eases the work of the admins.

[Ferdi265]

I also want to add that even if the mail server / domain is the same, the mail server will not necessarily allow you to send as some.user@my.tld with the credentials for the nextcloud notification mail (often something like nextcloud@my.tld).

Also, since normal non-admin users can usually add and change mail settings via the nextcloud mail app, whether overriding the From header with the notification mail server and credentials will work can even change easily just through a settings change by an unprivileged user!

So I would suggest:

• have a global check box in the admin panel:
  • [X] Allow sending Calendar Invitations from user mail addresses configured in the Mail app
• have a check box in the Mail app account settings (maybe disable / grey out this option if the global setting is disabled)
  • [X] Allow NextCloud to send Calendar Invititations from this address

and send out calendar invitations from the user mail address (using the user mail credentials!) only if both settings are enabled.

[JMoVS]

@ChristophWurst Has there been any progress on this? In my case, it seems that the emails sent by the server itself have a higher chance of ending in spam than the users' email addresses sending.

[gitterdoneplease]

@ChristophWurst Has there been any progress on this? In my case, it seems that the emails sent by the server itself have a higher chance of ending in spam than the users' email addresses sending.

I've been modifying the code for all of the versions since my first post to make this work. Would be great to have individual email addresses send calendar invites the way other systems work.

[MaLiPi10]

I also have a requirement and would be happy to help with this. At least a mail server with two domains. More could be added. Postfix + Dovecot + MySQL + Spamasassin + ClamAV.

Also need synchronizations and sending invitations to calendars on smartphones.

[gitterdoneplease]

Hi all, with SPF and DKIM records set correctly this really all works painlessly now except for manually changing the code. Could we please get something in the UI? It would be really great. Thanks so much.

[keunes]

with SPF and DKIM records set correctly this really all works painlessly now

Is there some guide that you followed?

[gitterdoneplease]

How to set DNS records correctly? There is a lot of help out there, but I’d look at MxToolBox.

[gitterdoneplease]

Every new version I come back here to find what needs to be changed again. It's still in nextcloudroot/apps/dav/lib/CalDAV/Schedule/IMipPlugin.php, but now the line number is 244.

Change line 244 from: ->setFrom([$fromEMail => $fromName]); to: ->setFrom([$sender => $senderName]);

[komoricodrutz]

Hi all, Yes, such a feature (selectable option from admin account) would indeed be beneficial. All calendar-related applications I've worked with do send the invitation from the user's logged in account.

And perhaps another small issue no-one seems to have noticed. At least not yet. When you send the appointment, it does not show up in the sent items mail folder. Neither in my user account's mailbox, nor in the admin account's mailbox (after I modified the setting as described in the comment above). Which can be a nuisance. Consider this scenario: If it doesn't show up in Sent Items, how am I to be sure that the invitation has actually been sent? Or how can I prove to my boss that I sent the appointment when I say I sent it? In addition, it doesn't always send the invites out. If your recipient has a nextcloud account on your own server and is registered with his private e-mail address, it seems that it doesnt' even send an e-mail. I performed these tests: I created an appointment with a friend's account, who has an account on my nextcloud and the registered e-mail address someone@gmail.com and my user account is someoneelse@gmail.com. someone@gmail.com never got the invite via e-mail. However, it arrived straight into his nextcloud calendar. And nothing in sent items. Then I only used just his someone@gmail.com address and again, he didn't receive the invite by mail. When I used his work e-mail address (someone@somethingelse.com), he finally received it. Yet still nothing in any profile sent items mailbox.

These "features" should not even be considered feature requests. They should simply be matching all the other similar solutions on the market. Out of the box.

LE, after performing some additional tests: It also appears that the solution from @gitterdoneplease Change line 244 from: ->setFrom([$fromEMail => $fromName]); to: ->setFrom([$sender => $senderName]); doesn't seem to work sufficiently. It only appears to modify the sender's displayname: I even rebooted the server after making this change and he is still receiving the appointment from the admin account's e-mail address (yahoo), albeit with my account's display name shown in the from field instead of the admin account's display name. May be related to the entry $fromEMail = Util::getDefaultEmailAddress('invitations-noreply'); I am fearful, however, to change this, as I am not versed at all in php. So maybe someone else can give me a tip in this context? @gitterdoneplease - pls elp? :-)

Currently tested on Nextcloud 25.0.12. Will test it soon on the latest version instance I have (27.1.2). If I don't post additional details, you can assume that the behaviour is the same.

My suggestion would be that if the currently logged in user has one or more e-mail account(s) configured in the "Mail" client interface, nextcloud should by default use the account configured as default in the mail client. The admin account should be left as a fallback solution, if the currently logged in user does not have an e-mail account configured in the mail client.

[keunes]

Hi @komoricodrutz

If your recipient has a nextcloud account on your own server and is registered with his private e-mail address, it seems that it doesnt' even send an e-mail. I performed these tests:

I think that'd deserve its own, dedicated issue. That'd probably more considered a bug than an enhancement/feature request.

[ChristophWurst]

See open work packages.

[susnux]

Question: Is it planned to deprecate the old symphony mailer abstraction by adding a system provider? Or should this just be an addition? Because now we have two ways to send emails.

[SebastianKrupinski]

Question: Is it planned to deprecate the old symphony mailer abstraction by adding a system provider? Or should this just be an addition? Because now we have two ways to send emails.

At the moment both code paths will exist. The mail provider at the moment will be used to solve the problem that because all messages up to this point have been sent from the system account, calendar invitation invite replies are never received by the user, and the system account can not be used to send messages on behalf of someone else especially if that user uses a different server.

But I have thought that we could use the mail provider to send most if not all user generated notification, lets say when a user shares a file with someone. And we could also implement the same mail provider for the system account that way when apps send a message both the user mail account and system account is available through the same interface/logic.

Hope that answers your question.

[susnux]

And we could also implement the same mail provider for the system account that way when apps send a message both the user mail account and system account is available through the same interface/logic.

Yes that is what I meant, if it is indendet to provide a system provider with e.g. "id = system" or capability "systemMail" or what ever.

Basically if this is a second way to sent mail, or the old way should be deprecated in favor of one consistent way handling mail :)

[kesselb]

I second the idea to make the default mailer an "mail provider".

We started with imip, but the same would be nice for sharebyemail, inviting guests users, other apps, etc.

[ChristophWurst]

Add an admin setting to choose which email system should be used (system or mail providers)

We do this so admins have the choice to "go back" to system emails for invites, either because they prefer that system or there is an issue with the mail providers.