virtdomains: userid is the only sane setting

[elliefm]

The virtdomains setting in imapd.conf has three possible values:

• off: Cyrus completely ignores domains altogether -- this is an ancient default. Incoming mail is delivered to the matching username account regardless of the recipient domain. Calendaring is non-functional because calendaring requires an account to have a single authoritative domain. It sort of uses defaultdomain to make up for lack of one, but this a) is untested and buggy (e.g. #2846), and b) cannot possibly accommodate setups where multiple domains are aliased or changeable outside Cyrus (e.g. DNS/MX), and so is precarious at best.
• userid: Cyrus knows about domains. Userids are expected to contain a domain (i.e. username@domain.tld). Delivery only occurs to the matching username@domain.tld. In this configuration, defaultdomain can be used to set a domain for unqualified usernames, with the caveat that username[@defaultdomain] and username@domain.tld are distinct accounts even if defaultdomain==domain.tld, so it causes headaches if you ever need to change it, and so relying on defaultdomain is not really recommended (and the default value is deliberately an undeliverable 'internal')
• on: Cyrus knows about domains. Userids may contain a domain, which will be respected. Bare usernames will have a domain magicked up via reverse DNS lookup on the IP address of the incoming network interface, and thus are liable to change from outside of Cyrus without Cyrus knowing, which breaks everything.

The ONLY sane setting, really, is virtdomains: userid.

From this, a few actions are evident:

• [ ] Our documentation needs to be written from the point of view that this is the only setting a new Cyrus install should even consider (even if they have no intention of ever supporting multiple domains), and that existing installations should aim to switch to this configuration at some point (even if they don't currently support multiple domains, and especially if they do)
• [ ] The default value for virtdomains needs to be changed to userid, not off. This change is not suitable for backporting to an existing stable release, so it can only happen on master, and whichever new stable series gets it will need clear upgrade documentation about it.
• [ ] At some point, we should remove the virtdomains config option entirely, and have the existing userid behaviour be the only behaviour. This will resolve the current development/maintenance headache of needing to consider and accommodate all the quirks of the other two values. A lot of old/confusing code can be cleaned up if we do this! It feels like a "4.0" scale of change rather than a "3.x", though.

[elliefm]

For 3.2, I would like our upgrade documentation to strongly encourage admins start migrating (or planning to eventually migrate) their systems to virtdomains=userid, so that we can more easily make it the default in some future release, and eventually make it the only behaviour without an option at all.

To do that, I'd like to have specific documentation for "how to convert your virtdomains=off or virtdomains=on system to use virtdomains=userid". I think most of the mess is going to be in renaming/migrating user accounts from userid to userid@domain.tld, and doing so in concert with updating authentication mechanisms, and local configurations for IMAP users.

But I've never done this, nor seen it done, and I don't know what other edge cases people will run into while doing this. (Will mupdate/notifyd choke? Will replication need special handholding? Who knows!) So I'm kind of stalled out because I don't feel expert enough to be doing the advising.

[hagedose]

To do that, I'd like to have specific documentation for "how to convert your virtdomains=off or virtdomains=on system to use virtdomains=userid". I think most of the mess is going to be in renaming/migrating user accounts from userid to userid@domain.tld, and doing so in concert with updating authentication mechanisms, and local configurations for IMAP users.

I'd very much like to see this myself, and perhaps I can help . We're still running 2.4 without virtdomains. If we use replication for upgrading, can we enable virtdomains=userid on the new server before we start the replication process? If not, I could replicate a few accounts to have a playground where I can see what happens if I change a system from virtdomains=off to virtdomains=uid.

[elliefm]

If we use replication for upgrading, can we enable virtdomains=userid on the new server before we start the replication process?

That's a really good question. I actually have no idea -- usually I would say "yes, this is fine, because the replication format is mostly agnostic of the underlying storage" -- but, the virtdomains options affect peoples userid's, and I'm not sure if this will affect how they replicate.

If you're able to give it a go and report back, that'd be really useful to know (and document!)

If not, I could replicate a few accounts to have a playground where I can see what happens if I change a system from virtdomains=off to virtdomains=uid.

Heh, I guess there'll be replication happening anyway. Might as well try it the "easy way" first, and see if it Just Works. But it would also be very useful to see what's required to monkeypatch a system where virtdomains has been changed.

I'd very much like to see this myself, and perhaps I can help.

Would be greatly appreciated! :)

[hagedose]

My first tests are very promising. I set virtdomains to userid and defaultdomain to uni-koeln.de. As long as userids are used either unqualified or with the default domain, there is absolutely no difference to virtdomains=off. I've seen that the virtdomains code treats local parts with dots differently, but we don't have any. So I went ahead and synced an account from 2.4 to 3.2, and it works just fine.

But I believe the documentation regarding defaultdomain is outdated: https://www.cyrusimap.org/3.2/imap/concepts/features/virtual-domains.html

Even for new installations, set the defaultdomain to the “real” domain of the server (domain of its primary hostname).

I think that recommendation only makes sense in the context of virtdomains=on, where the domain may depend on the IP address of the receiving interface. In our case the defaultdomain is mostly irrelevant, at least for now.

I played around with users in additional domains, and with our authentication setup (ldapdb auxprop) that leads to a surprising result. The user cannot be authenticated (fine), but it doesn't even appear in the log. Instead it is given as -notset-:

Mai 18 08:38:57 cyrus3-centos8.rrz.uni-koeln.de cyrus/imap[5283]: badlogin: localhost [::1] DIGEST-MD5 (-notset-) [SASL(-13): user not found: unable to canonify user and get auxprops]

Is that normal behavior?

To summarize, if your usernames don't contain dots, it should be possible to switch from virtdomains=off to virtdomains=userid without problems.

[elliefm]

I mentioned in the original post,

In this configuration, defaultdomain can be used to set a domain for unqualified usernames, with the caveat that username[@defaultdomain] and username@domain.tld are distinct accounts even if defaultdomain==domain.tld, so it causes headaches if you ever need to change it, and so relying on defaultdomain is not really recommended (and the default value is deliberately an undeliverable 'internal')

I set virtdomains to userid and defaultdomain to uni-koeln.de.

So, the success behaviour you're seeing is based on defaultdomain being set to that domain, which is okay as long as it never needs to change, and as long as there's never any other domains. But if it ever needs to change, or if you introduce additional domains, life will get very messy. This isn't the approach we would recommend long term, but since it works smoothly for the trivial "switch virtdomains=off to virtdomains=userid" case, it might be a viable interim step....

I played around with users in additional domains, and with our authentication setup (ldapdb auxprop) that leads to a surprising result. The user cannot be authenticated (fine), but it doesn't even appear in the log. Instead it is given as -notset-:

My gut feel is that this might be related to defaultdomain. Note that users in other domains MUST authenticate as their fully-qualified userid@domain.tld account name for Cyrus to find their account. If you haven't changed it yet, your LDAP configuration probably still expects users to always login as userid and therefore maybe the LDAP bind succeeds, but then Cyrus can't find the account? Or the LDAP bind fails, because the wrong attribute value was used.

I think it's very important to stress here (and, eventually, in the documentation) that, under virtdomains=user:

• a user whose domain is not defaultdomain MUST authenticate as userid@domain.tld (and the authentication mechanism must be configured to recognise them as such)
• a user whose domain is the defaultdomain MAY authenticate as either userid OR userid@domain.tld (provided the authentication mechanism is configured to recognise either) BUT will be accessing a different account in Cyrus depending on which they use. (Now: imagine this user configures a new client, with the wrong version of their userid: it works, but "all my mail is gone!")

Which is why the default value for defaultdomain is the undeliverable value "internal". It does not make any sense to use defaultdomain at all, really, and I think I would consider this setting to be on the long term kill-list along with virtdomains.

But! If you're not using defaultdomain=[your original domain] when migrating from virtdomains=off to virtdomains=userid, then you MUST rename every account from userid to userid@domain.tld during the process. And fix every user's client configuration to authenticate with their fully-qualified account name instead of just the userid they've been using all these years. Ouch!

[hagedose]

I mentioned in the original post,

In this configuration, defaultdomain can be used to set a domain for unqualified usernames, with the caveat that username[@defaultdomain] and username@domain.tld are distinct accounts even if defaultdomain==domain.tld, so it causes headaches if you ever need to change it, and so relying on defaultdomain is not really recommended (and the default value is deliberately an undeliverable 'internal')

I do not believe that is true, based on my observations.

localhost> cm user/xxx
localhost> cm user/xxx@uni-koeln.de
createmailbox: Mailbox already exists

I can use xxx and xxx@uni-koeln.de interchangeably.

imtest -u a0620 -a a0620 localhost
S: * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE STARTTLS LOGINDISABLED AUTH=DIGEST-MD5 AUTH=CRAM-MD5 AUTH=NTLM SASL-IR] cyrus3-centos8.rrz.uni-koeln.de Cyrus IMAP 3.2.0-2.el8 Fedora server ready
C: A01 AUTHENTICATE DIGEST-MD5
S: + ...=
Please enter your password:
C: ...
S: + ...==
C:
S: A01 OK Success (privacy protection) SESSIONID=<cyrus-1589974092-12148-1-4954642977666001847>
Authenticated. 

imtest -u a0620@uni-koeln.de  -a a0620@uni-koeln.de localhost
S: * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE STARTTLS LOGINDISABLED AUTH=DIGEST-MD5 AUTH=CRAM-MD5 AUTH=NTLM SASL-IR] cyrus3-centos8.rrz.uni-koeln.de Cyrus IMAP 3.2.0-2.el8 Fedora server ready
C: A01 AUTHENTICATE DIGEST-MD5
S: + ...=
Please enter your password:
C: ...==
S: + ...==
C:
S: A01 OK Success (privacy protection) SESSIONID=<cyrus-1589974115-10199-1-14658711718514446620>
Authenticated.
Security strength factor: 128

In mailboxes.db there is only ever an unqualified entry when I create a mailbox with the defaultdomain appended. The only mailboxes that have the domain added are those that are created with a domain other than the defaultdomain.

I played around with users in additional domains, and with our authentication setup (ldapdb auxprop) that leads to a surprising result. The user cannot be authenticated (fine), but it doesn't even appear in the log. Instead it is given as -notset-:

My gut feel is that this might be related to defaultdomain. Note that users in other domains MUST authenticate as their fully-qualified userid@domain.tld account name for Cyrus to find their account. If you haven't changed it yet, your LDAP configuration probably still expects users to always login as userid and therefore maybe the LDAP bind succeeds, but then Cyrus can't find the account? Or the LDAP bind fails, because the wrong attribute value was used.

ldapdb doesn't use the user for bind, but I guess it's possible that the reason is in the SASL layer. I have some ideas I'm going to try.

I think it's very important to stress here (and, eventually, in the documentation) that, under virtdomains=user:

• a user whose domain is not defaultdomain MUST authenticate as userid@domain.tld (and the authentication mechanism must be configured to recognise them as such)

Understood.

• a user whose domain is the defaultdomain MAY authenticate as either userid OR userid@domain.tld (provided the authentication mechanism is configured to recognise either) BUT will be accessing a different account in Cyrus depending on which they use. (Now: imagine this user configures a new client, with the wrong version of their userid: it works, but "all my mail is gone!")

Again, I do not believe that is true. If your system behaves differently, there must be some difference in the configuration, I guess.

[elliefm]

Oh, that's very interesting about defaultdomain. I need to do some testing, and figure out where I learned the "these are distinct accounts" behaviour from. It might be something I learned for 2.4 or 2.5 (or maybe even 3.0) that's since been fixed. Or, it might be dependent on some specific configuration, dunno! Thanks for the correction :)

[elliefm]

This will also require a bunch of Cassandane work, since most of our tests expect it to be off by default, and use the :VirtDomains magic to set it to userid if they care about it being enabled. So we'd need to invert the sense of all these, or review and maybe amend all the default-case ones to expect the virtdomains: userid behaviour