CalDAV invite is being sent from "@internal" domain

[tntclaus]

I've got 3.0.8 installation with caldav support on the Debian 10.

CalDAV is working, users can authenticate and create/update/delete personal events, but when they try to send an invitation to, say, someone@gmail.com, the invitation is sent not from user@domain.tld but from user@internal. Such invitation is pointless and can't be accepted. All other emails are sent with correct From addresses.

Machine hostname is set to domain.tld. Users authenticate to HTTPS/IMAPS with username (not username@domain.tld) and password credentials (this is single domain mail server). virtdomains feature disabled (enabling this makes things even worse: caldav will not send invitation emails at all).

Is it a bug or did I misconfigure something?

[elliefm]

It sounds like you have defaultdomain in imapd.conf at its default value of "internal"? This is the domain that is used for users who are logged in as "username" (rather than "username@domain.tld")

Generally, if you only have a single domain, I think your options are either:

• leave defaultdomain at its default value of "internal", and have users always and only log in using their full "username@domain.tld" (never just "username")
• set defaultdomain to your domain.tld and have users log in using just "username" (and never "username@domain.tld")

I'm not completely sure of the ramifications of either configuration, or which is best practice. Also, note that "username" and "username@domain.tld" are distinct accounts from Cyrus's perspective (even if defaultdomain=domain.tld), so if you naively switch schemes around on a live system, users will suddenly be logging into a "different" (probably empty!) account.

I think whether or not you anticipate adding additional domains later might have some bearing on which scheme is best. You might want to ask about this on the info-cyrus mailing list?

[tntclaus]

@elliefm thank you for your reply. I didn't make it clear, but with choosing not to use virtdomains setting in imapd.conf I as well do not set defaultdomain setting as it should work (according to doc ^_^ ) only with virtdomains: on.

I already tried to set defaultdomain to my domain.tld (tried both casesvirtdomains: on and virtdomains unset), but it led to situation when caldav decided not send invitation emails anymore.

[elliefm]

The reason I'm looking so intently at defaultdomain is that that's the only place that the string 'internal' is likely to come from... If it's using that value when documentation says it shouldn't, that's starting to sound like a bug, either with the documentation or the implementation!

I'm having a look at Cassandane, and all of the invite tests have virtdomains: on. I'm not sure if this is meant to imply that CalDAV invites are only supposed to work in virtdomains environments, or if the complementary tests without it just never got written.

I know there's been a lot of calendaring work done on master that's been too big/intrusive to backport to 3.0, so this might be already fixed on master, but the lack of any tests implies not!

I'll need to look further into this to find out how it's supposed to work...

[brong]

Virtdomains is a tristate, and "on" does not mean the same thing as "userid". Cassandane has "userid", which is what we run at Fastmail and is really the only option that makes sense for caldav, since caldav invitations don't handle the concept of aliases at all well, and other virtdomain settings act like aliases on a machine with multiple domains.

[elliefm]

Okay so there's one documentation bug: userid is listed as one of the allowed virtdomains values, but there's no explanation as to what it does!

virtdomains: off

Enable virtual domain support. If enabled, the user’s domain will be determined by splitting a fully qualified userid at the last ‘@’ or ‘%’ symbol. If the userid is unqualified, and the virtdomains option is set to “on”, then the domain will be determined by doing a reverse lookup on the IP address of the incoming network interface, otherwise the user is assumed to be in the default domain (if set). Allowed values: off, userid, on

Also, that "if set" is misleading these days, because it dates back to when defaultdomain defaulted to NULL, but that is no longer the case, so it's always "set".

https://cyrusimap.org/imap/concepts/features/virtual-domains.html kind of talks about this stuff, but this is a hard page to find!

I haven't found much documentation for calendaring at all really... seems to be a "figure it out yourself, then never tell anyone" kinda feature!

It kind of sounds like, when virtdomains=off, the calendar is expecting to find the domain part in defaultdomain, which seems like a reasonable (if undocumented) fall back position (since calendaring 100% needs a domain, unlike imap). But we haven't got any tests for that configuration, and it sounds like it's currently inconsistent (as reported: no invitations at all when set to an actual domain, bad invitations when defaulted to 'internal').

I think I'm gonna write some tests specifically for virtdomains=off and see what it does. I'm still not sure if this configuration is thereotically possible or not (ignoring bugs), but I'm not gonna learn that from reading nonexistent docs!

@tntclaus as regards the real-world problem of getting this working with debian's 3.0.8:

It kind of sounds like, at present, you'd need to switch to virtdomains=userid and have users log in as username@domain.tld (including migrating data from the username mailboxes to username@domain.tld mailboxes... uggh).

It might work the way you want if you switched to a build from git master, but it's untested, so it might not. And if it doesn't work now, it might work eventually, but even if it's fixed and backported to 3.0, it would still require an upgrade on your part when it does. :(

[elliefm]

I've done some more digging offline and it sounds like virtdomains: userid is the only sensible setting anymore (thus #2847), and will probably become the default and/or only option in the future. So, as much of a nuisance as it may be to reconfigure for that, it's probably the most future-compatible action, and it'll solve your immediate problem too.

I'm not really fully across the details of reconfiguring in this way, but off the top of my head, some stuff you'll definitely need to take into consideration:

• the aforementioned distinction between "username" and "username@domain.tld" accounts: preferably switch to having no bare-username accounts (and leave defaultdomain at its default of "internal")
• your lmtp/mx delivery setup (esp while you still have bare-username accounts)
• your users' mail client configurations (they might need to change their credential from bare-username to username@domain.tld)
• your auth setup (esp if existing users want to continue to login as bare-username, maybe your auth handler can translate this to their real username@domain.tld)

I think there is some documentation for this process somewhere, but I'm not sure where?

[tntclaus]

@elliefm thanks a lot for your research. I will follow your recommendations and will come back to drop a comment.

After that I can try to write/update quickstart documentation for Cyrus (imaps/caldav/carddav) + Postfix server setup with sasldb2 (which I have and which behavior seems is a bit odd for me) or ldap/radius (which I plan to setup by the end of the week) authentication backends. Right now such documentation is scattered accross different webpages written for different, sometimes outdated, versions.

[tntclaus]

Ok, I've got this up and running with LDAP authentication (actually it is using PAM against freeipa, but technically data is stored in LDAP ;) ). I also found some issues with caldav authentication: it doesn't work without SSL at least with OS X Calendar.app :)

As I had said I can prepare a manual how to setup cyrus imap + postfix + caldav with authentication against corporate identity backend. It turns out the backend will be freeipa and auth mech will be PAM.

Please tell me if it is interesting and will be useful for community.

[elliefm]

If you frame it as being a CalDAV guide (with PAM/LDAP being coincidental details), I think that might be pretty useful! I think this probably would fit under Operations/Administrator Guide/Management, in the same way as the "Cyrus Backups", "Cyrus Sieve" and "Cyrus NNTP" features are documented?

A pull request would be amazing, but if you'd rather just write plain text or markdown, I'm happy to massage it into rst format

[mpghf]

For anybody else coming across this: I came across the same thing. To work around it, I used:

http_virtdomains: userid http_defaultdomain: mydomain.com caldav_realm: myrealm

where mydomain.com is the domain I want attached to the end of the username for calendar events, and myreal is what I want for SASL to use. That allows my users to continue logging in to http/CalDAV/CardDAV as they were previously, using the same URLs as previously, but getting the correct domain attached to their address when creating events.

(using version 3.4.4)