Can't use pts_module: ldap when also using ldapdb for SASL

[hagedose]

I was fascinated by the setup described in #3115 because we are in the same boat as @akschu. Currently our users have to use their UNIX account name to log in, and they can't use their email addresses. I managed to get ptloader to work based on their configuration example, but only under very specific conditions:

I can't get it to work with 2.4.20 at all. I can see that ptloader successfully queries the LDAP server, but then something goes wrong. I understand that 2.4 isn't supported anymore, so I don't worry about that.

I can get it to work with 3.2.2, but:

• The email address must not be in the defaultdomain
• Authentication must not use ldapdb as the auxprop mechanism

I suppose the first is a non-issue, because if all valid email addresses are successfully canonified we don't need an actual defaultdomain anymore. Is that a correct assumption?

The second is a huge problem for us. A sanitized version of our ideal configuration is this:

sasl_auxprop_plugin: ldapdb
sasl_ldapdb_uri: ldaps://mailauth.example.com
sasl_ldapdb_id: mailproxy
sasl_ldapdb_pw: secret
sasl_ldapdb_mech: DIGEST-MD5

pts_module: ldap
auth_mech: pts
ldap_base: ou=People,dc=example,dc=com
ldap_sasl: no
ldap_bind_dn: cn=mailmgr,ou=DSA,dc=example,dc=com
ldap_password: secret
ldap_uri: ldaps://maillookup.example.com
ldap_filter: (|(uid=%u)(mail=%u))
ldap_user_attribute: uid
ptloader_sock: /var/lib/imap/socket/ptsock
ldap_size_limit: 100
ldap_member_method: attribute
ldap_member_attribute: member

So there are two separate servers with different ids. When I try the setup as given, ptloader gets confused. It successfully canonifies the address, but then this happens:

cyrus/imap[113467]: ptload completely failed: unable to canonify identifier: mailproxy

Somehow the id used for ldapdb ends up in ptloader.

[akschu]

I suppose the first is a non-issue, because if all valid email addresses are successfully canonified we don't need an actual defaultdomain anymore. Is that a correct assumption?

Yes, I don't use virtdomains:

virtdomains: no

The second is a huge problem for us. A sanitized version of our ideal configuration is this:

sasl_auxprop_plugin: ldapdb
sasl_ldapdb_uri: ldaps://mailauth.example.com
sasl_ldapdb_id: mailproxy
sasl_ldapdb_pw: secret
sasl_ldapdb_mech: DIGEST-MD5

pts_module: ldap
auth_mech: pts
ldap_base: ou=People,dc=example,dc=com
ldap_sasl: no
ldap_bind_dn: cn=mailmgr,ou=DSA,dc=example,dc=com
ldap_password: secret
ldap_uri: ldaps://maillookup.example.com
ldap_filter: (|(uid=%u)(mail=%u))
ldap_user_attribute: uid
ptloader_sock: /var/lib/imap/socket/ptsock
ldap_size_limit: 100
ldap_member_method: attribute
ldap_member_attribute: member

So there are two separate servers with different ids. When I try the setup as given, ptloader gets confused. It successfully canonifies the address, but then this happens:

cyrus/imap[113467]: ptload completely failed: unable to canonify identifier: mailproxy

Somehow the id used for ldapdb ends up in ptloader.

I have a single server, so much of your configuration includes things I haven't spent time understanding.

Here is my config:

# usernames and auth
lmtp_downcase_rcpt: yes
username_tolower: yes
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
allowapop: no
allowplaintext: yes
virtdomains: no
pts_module: ldap
auth_mech: pts
ldap_base: dc=domain,dc=net
ldap_sasl: no
ldap_bind_dn: cn=Manager,dc=domain,dc=net
ldap_password: secret
ldap_uri: ldap://localhost
ldap_filter: (|(uid=%u)(mailLocalAddress=%u))
ldap_user_attribute: uid
ptloader_sock: /var/spool/imap/socket/ptsock
ldap_size_limit: 100
ldap_member_method: attribute
ldap_member_attribute: member

On the sendmail side (haven't looked at postfix yet) I resolve all email addresses to users with a virtusertable map.

FEATURE(virtusertable',ldap:-1 -k (&(objectClass=inetLocalMailRecipient)(mailLocalAddress=%0)) -v mailRoutingAddress')

The mailRoutingAddress ether has another email address in the case of forwarding or the username which causes sendmail to do lmtp.

Sendmail uses standard sasl authentication and saslauthd looks like this:

ldap_auth_method: bind
ldap_servers: ldap://127.0.0.1/
ldap_filter: (|(uid=%u)(mailLocalAddress=%u))
ldap_version: 3
ldap_search_base: dc=domain,dc=net
ldap_scope: sub
ldap_use_sasl: no

That way it finds the user either way and whatever DN that comes back is what we bind to for authentication.

This might break relaying in sendmail (I can't remember) but I don't do much processing in sendmail anymore. I use a milter called mimedefang and have a fairly elaborate filter which blocks relaying as well as pulls other attributes from ldap which can then figure out if the email message is sourced from an address other than the email addresses attached to the authenticated account (spoofing) and block those, and about 50 other things...

Hope that helps.

[elliefm]

I can't get it to work with 2.4.20 at all. I can see that ptloader successfully queries the LDAP server, but then something goes wrong. I understand that 2.4 isn't supported anymore, so I don't worry about that.

We got some patches in the last couple of years which fixed this particular behaviour up, but I can't remember which version they were for. Maybe 2.5? It sounds like 2.4 didn't get them, anyway. Good to know.

I can get it to work with 3.2.2, but:

• The email address must not be in the defaultdomain

I think this is what I'd expect. I think we've talked before about the differences between user[@defaultdomain] and user@an-actual.domain? ptloader being confused by it is probably the least surprising thing I'll read all week. :)

• Authentication must not use ldapdb as the auxprop mechanism

I suppose the first is a non-issue, because if all valid email addresses are successfully canonified we don't need an actual defaultdomain anymore. Is that a correct assumption?

Yep. In fact, this is the configuration we recommend for new deployments -- though it looks like our documentation is out of date, doh. Nowadays, "defaultdomain" has a default value (so you're no longer forced to set it yourself when setting up a Cyrus deployment), and that default value is "internal". The idea is that every real user should be fully-qualified in Cyrus. Calendaring needs this, among other things. Internal accounts that don't receive mail, like "cyrus" (or whatever else you use for administration) can be left unqualified, and won't be externally-deliverable (unless your SMTP setup aliases something@deliverable -> userid@internal).

The second is a huge problem for us. A sanitized version of our ideal configuration is this:

sasl_auxprop_plugin: ldapdb
sasl_ldapdb_uri: ldaps://mailauth.example.com
sasl_ldapdb_id: mailproxy
sasl_ldapdb_pw: secret
sasl_ldapdb_mech: DIGEST-MD5

pts_module: ldap
auth_mech: pts
ldap_base: ou=People,dc=example,dc=com
ldap_sasl: no
ldap_bind_dn: cn=mailmgr,ou=DSA,dc=example,dc=com
ldap_password: secret
ldap_uri: ldaps://maillookup.example.com
ldap_filter: (|(uid=%u)(mail=%u))
ldap_user_attribute: uid
ptloader_sock: /var/lib/imap/socket/ptsock
ldap_size_limit: 100
ldap_member_method: attribute
ldap_member_attribute: member

So there are two separate servers with different ids. When I try the setup as given, ptloader gets confused. It successfully canonifies the address, but then this happens:

cyrus/imap[113467]: ptload completely failed: unable to canonify identifier: mailproxy

Somehow the id used for ldapdb ends up in ptloader.

I don't know much about how auxprop/sasl work in detail. I vaguely remember there having been some problem with the DIGEST-MD5 mechanism, but I can't remember what it was, where it was, or whether it was fixed. If you're able to use a different sasl_ldapdb_mech instead, does doing so make any difference?

Though I think the real problem here is ptloader getting confused, so don't sweat it just now if you can't change the mechanism. I don't know why ptloader would be getting confused (it shouldn't even see the sasl_ config options??), but once again I'm kind of unsurprised that it is.

I'm curious about your setup whereby separate servers (or at least: separate hostnames) are used for lookups vs auth. Is that a common setup in the LDAP world? I'm familiar with large LDAP deployments, but the last time I was near one is coming up on a decade ago, so that familiarity is both rusty and probably getting obsolete. I would maybe expect separate servers if the auth server was LDAPS and the query server wasn't, but both of yours are LDAPS, so that's clearly not the motivation.

I wonder if ptloader is trying to be clever, and is reusing the sasl configuration instead of (or as well as) its own?

[hagedose]

The second is a huge problem for us. A sanitized version of our ideal configuration is this:

sasl_auxprop_plugin: ldapdb
sasl_ldapdb_uri: ldaps://mailauth.example.com
sasl_ldapdb_id: mailproxy
sasl_ldapdb_pw: secret
sasl_ldapdb_mech: DIGEST-MD5

pts_module: ldap
auth_mech: pts
ldap_base: ou=People,dc=example,dc=com
ldap_sasl: no
ldap_bind_dn: cn=mailmgr,ou=DSA,dc=example,dc=com
ldap_password: secret
ldap_uri: ldaps://maillookup.example.com
ldap_filter: (|(uid=%u)(mail=%u))
ldap_user_attribute: uid
ptloader_sock: /var/lib/imap/socket/ptsock
ldap_size_limit: 100
ldap_member_method: attribute
ldap_member_attribute: member

So there are two separate servers with different ids. When I try the setup as given, ptloader gets confused. It successfully canonifies the address, but then this happens: cyrus/imap[113467]: ptload completely failed: unable to canonify identifier: mailproxy Somehow the id used for ldapdb ends up in ptloader.

I don't know much about how auxprop/sasl work in detail. I vaguely remember there having been some problem with the DIGEST-MD5 mechanism, but I can't remember what it was, where it was, or whether it was fixed. If you're able to use a different sasl_ldapdb_mech instead, does doing so make any difference?

No, but it was worth a try.

Though I think the real problem here is ptloader getting confused, so don't sweat it just now if you can't change the mechanism. I don't know why ptloader would be getting confused (it shouldn't even see the sasl_ config options??), but once again I'm kind of unsurprised that it is.

I'm curious about your setup whereby separate servers (or at least: separate hostnames) are used for lookups vs auth. Is that a common setup in the LDAP world?

I don't run the LDAP servers, but I can say that in our case it's a security issue. The LDAP server that's used for ldapdb contains password equivalent hashes so that it can be used for challenge-response mechanisms such as DIGEST-MD5 and SCRAM. So it needs more protection than the server used to look up the email addresses for the accounts we have.

I wonder if ptloader is trying to be clever, and is reusing the sasl configuration instead of (or as well as) its own?

It's strange because the ptloader part works. Here is a sanitized but complete log of a failed login attempt when both LDAP servers are enabled:

Jul 27 17:09:35 cyrus3-centos8 cyrus/imap[117247]: command: A01 Authenticate
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: ptload(): fetched cache record (myname@example.com)(mark 1595590200, current 1595862578, limit 1595851778)
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: ptload(): pinging ptloader
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: connected with no delay
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: ptload(): connected
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: timeout_select: sock = 11, rp = 0x0, wp = 0x7ffdde911c40, sec = 30
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: timeout_select exiting. r = 1; errno = 0
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: ptload sent data
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: timeout_select: sock = 11, rp = 0x7ffdde911bc0, wp = 0x0, sec = 30
Jul 27 17:09:38 cyrus3-centos8 cyrus/ptloader[117235]: accepted connection
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: timeout_select exiting. r = 1; errno = 0
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: timeout_select: sock = 11, rp = 0x7ffdde911bc0, wp = 0x0, sec = 30
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: timeout_select exiting. r = 1; errno = 0
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: ptload read data back
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: ptload returning data
Jul 27 17:09:38 cyrus3-centos8 cyrus/imap[117247]: canonified myname@example.com -> myuid
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: ptload(): pinging ptloader
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: connected with no delay
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: ptload(): connected
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: timeout_select: sock = 12, rp = 0x0, wp = 0x7ffdde9115b0, sec = 30
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: timeout_select exiting. r = 1; errno = 0
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: ptload sent data
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: timeout_select: sock = 12, rp = 0x7ffdde911530, wp = 0x0, sec = 30
Jul 27 17:09:39 cyrus3-centos8 cyrus/ptloader[117235]: accepted connection
Jul 27 17:09:39 cyrus3-centos8 cyrus/ptloader[117235]: No entries found
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: timeout_select exiting. r = 1; errno = 0
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: timeout_select: sock = 12, rp = 0x7ffdde911530, wp = 0x0, sec = 30
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: timeout_select exiting. r = 1; errno = 0
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: ptload read data back
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: ptload(): bad response from ptloader server: identifier not found
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: No data available at all from ptload()
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: ptload completely failed: unable to canonify identifier: mailproxy
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: SASL bad userid authenticated
Jul 27 17:09:39 cyrus3-centos8 cyrus/imap[117247]: badlogin: localhost [::1] CRAM-MD5 (myuid) [SASL(-1): generic failure: security flags do not match required]

Now that I'm looking at it, I have a suspicion: it looks as though ptloader is asked twice, once for the actual login, and once for the account that's used by the proxy authentication mechanism. The account mailproxy can't be found in the LDAP server that's used by ptloader. Does that help? FWIW, I used imtest with a variety of mechanisms to rule out that it makes a difference.

[elliefm]

Sooo, let me see if I'm understanding correctly:

• "mailproxy" is a user that exists in the mailauth instance only
• to verify an imap user's credentials in mailauth, the lookup must be authenticated as the "mailproxy" user, which is configured by the sasl_* options

And it looks like ptloader talks to maillookup to resolve (correctly) that "myname@example.com" is "myuid". And then having this information, it wants to authenticate. But in order to talk to mailauth to authenticate, it thinks it needs to resolve "mailproxy" to something first, and can't because maillookup doesn't know about it.

Does that match your current understanding?

Though you mention proxy authentication, so I wonder if the "mailproxy" that ptloader complains about is coming from the "sasl_ldabdb_id" configuration (as I have so far assumed, and which seems very weird), or whether it's coming from your imtest invocation. Are you able to provide a sanitised copy of the imtest output (including the invocation)?

A bit of context as to what informs my mental model here:

In the olden days, when I ran a (non-Cyrus) mail system backed by an LDAP directory (like you, it was managed by a separate team), our IMAP servers didn't use SASL. The login flow looked like this:

1) client provides username/password pair (over an SSL or STARTTLS connection) 2) IMAP server looks up the username in the LDAP directory to find the user's DN. I think the filter used was something like "(&(objectclass=mailuser)(uid=%s))", or something to that effect. 3) IMAP server attempts to BIND to the directory using the discovered DN and the provided password 4) if the BIND succeeds, it trusts that the user is who they say they are, and lets them in

(I'm glossing over the authentication/authorisation distinction a little here, i.e. the -a and -u imtest options, but it worked the same way: if you provided a bindable username/password pair for a user with permission to be someone else, then you could be someone else.)

The "username" in 1 & 2 was a uid, but I think it could just as easily have been an email address, or either a uid or email address. It would've just meant changing the filter used to look up the DN. At the time, logging in with a uid was what one did, so that was what we did.

This is still how I imagine LDAP-based authentication usually functions, and I think might be how ptsloader expects it to work. This might explain why ptsloader is trying to look up "mailproxy": maybe it's trying to convert that into a DN to bind with? Your setup seems strange to me, but I can't tell if it's actually strange, or if what I'm accustomed to is obsolete.

We did use SASL for authentication to our SMTP servers (postfix + cyrus-sasl), and the SASL configuration used the same LDAP directory under the hood, but I have no recollection of what our configuration looked like or how it worked... I didn't really understand SASL back then, and still kinda don't. But I believe/assume it used a similar premise, i.e. just take a simple username/password pair and verify it by trying to BIND with it.

[hagedose]

Sooo, let me see if I'm understanding correctly:

• "mailproxy" is a user that exists in the mailauth instance only
• to verify an imap user's credentials in mailauth, the lookup must be authenticated as the "mailproxy" user, which is configured by the sasl_* options

And it looks like ptloader talks to maillookup to resolve (correctly) that "myname@example.com" is "myuid". And then having this information, it wants to authenticate. But in order to talk to mailauth to authenticate, it thinks it needs to resolve "mailproxy" to something first, and can't because maillookup doesn't know about it.

Does that match your current understanding?

Yes.

Though you mention proxy authentication, so I wonder if the "mailproxy" that ptloader complains about is coming from the "sasl_ldabdb_id" configuration (as I have so far assumed, and which seems very weird), or whether it's coming from your imtest invocation. Are you able to provide a sanitised copy of the imtest output (including the invocation)?

I don't think that will help, because it looks like any other imtest invocation:

 imtest -u myname@example.com  -a myname@example.com localhost
S: * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE STARTTLS LOGINDISABLED AUTH=DIGEST-MD5 AUTH=CRAM-MD5 AUTH=NTLM SASL-IR] cyrus.example.com Cyrus IMAP 3.2.2-dirty-1.el8 Fedora server ready
C: A01 AUTHENTICATE DIGEST-MD5
S: + ...=
Please enter your password:
C: ...==
S: A01 NO generic failure
Authentication failed. generic failure
Security strength factor: 128

When I disable ldapdb and authenticate against a local sasldb file, that same invocation works.

A bit of context as to what informs my mental model here:

In the olden days, when I ran a (non-Cyrus) mail system backed by an LDAP directory (like you, it was managed by a separate team), our IMAP servers didn't use SASL. The login flow looked like this:

1. client provides username/password pair (over an SSL or STARTTLS connection)
2. IMAP server looks up the username in the LDAP directory to find the user's DN. I think the filter used was something like "(&(objectclass=mailuser)(uid=%s))", or something to that effect.
3. IMAP server attempts to BIND to the directory using the discovered DN and the provided password
4. if the BIND succeeds, it trusts that the user is who they say they are, and lets them in

(I'm glossing over the authentication/authorisation distinction a little here, i.e. the -a and -u imtest options, but it worked the same way: if you provided a bindable username/password pair for a user with permission to be someone else, then you could be someone else.)

The "username" in 1 & 2 was a uid, but I think it could just as easily have been an email address, or either a uid or email address. It would've just meant changing the filter used to look up the DN. At the time, logging in with a uid was what one did, so that was what we did.

This is still how I imagine LDAP-based authentication usually functions, and I think might be how ptsloader expects it to work. This might explain why ptsloader is trying to look up "mailproxy": maybe it's trying to convert that into a DN to bind with? Your setup seems strange to me, but I can't tell if it's actually strange, or if what I'm accustomed to is obsolete.

No, it's not obsolete, it's just a different model, and definitely the more common one. It has one downside: that way you can't use challenge-response based authentication mechanisms. If you want to use those (APOP, NTLM, CRAM-MD5, DIGEST-MD5, SCRAM), you can't just bind to the LDAP server. You need either a local sasldb or the ldapdb feature.

The ldapdb model is best described here: https://openldap.org/doc/admin24/sasl.html

[elliefm]

I don't think that will help, because it looks like any other imtest invocation:

That was actually really helpful, because it confirmed that the "mailproxy" ptsloader is looking for must be coming from sasl_ldapdb_id, and not from an argument to imtest.

No, it's not obsolete, it's just a different model, and definitely the more common one. It has one downside: that way you can't use challenge-response based authentication mechanisms. If you want to use those (APOP, NTLM, CRAM-MD5, DIGEST-MD5, SCRAM), you can't just bind to the LDAP server. You need either a local sasldb or the ldapdb feature.

The ldapdb model is best described here: https://openldap.org/doc/admin24/sasl.html

That makes sense, thanks.

I don't really have a good hunch of what's going on here yet, nor whether the problem can be resolved by finding the correct configuration combination, or whether it requires code changes to achieve. We probably need to look at the line that's generating the "ptload completely failed: unable to canonify identifier: mailproxy" and trace backwards to find how and why it got there.

I don't think this will actually help, but I wonder what happens if you change sasl_ldapdb_id into an already-canonified form (whatever that might mean). Does it then just use it, without trying (and failing) to canonify it? Or does it naively still try to canonify it, and fail in the same way?

[hagedose]

I don't really have a good hunch of what's going on here yet, nor whether the problem can be resolved by finding the correct configuration combination, or whether it requires code changes to achieve. We probably need to look at the line that's generating the "ptload completely failed: unable to canonify identifier: mailproxy" and trace backwards to find how and why it got there.

I tried looking at the code, but I didn't get very far. If you can give me pointers where I could sensibly set a breakpoint, I could try using gdb to find out more.

I don't think this will actually help, but I wonder what happens if you change sasl_ldapdb_id into an already-canonified form (whatever that might mean). Does it then just use it, without trying (and failing) to canonify it? Or does it naively still try to canonify it, and fail in the same way?

Well, it's literally mailproxy, so there isn't anything to be canonified. I was thinking about adding mailproxy to the LDAP server that ptloader queries. Maybe that would be a workaround. I'll have to talk to the people running the LDAP server for that.

[elliefm]

I tried looking at the code, but I didn't get very far. If you can give me pointers where I could sensibly set a breakpoint, I could try using gdb to find out more.

Sure. I'll have a look myself, and when I get to a point where I need actual evidence rather than just code, I'll let you know where and what I need. I might also end up sending you a patch with a whole lot more debug logging (though ptloader is pretty verbose already, so maybe it won't be needed).

Well, it's literally mailproxy, so there isn't anything to be canonified.

Ahh

I was thinking about adding mailproxy to the LDAP server that ptloader queries. Maybe that would be a workaround. I'll have to talk to the people running the LDAP server for that.

I was thinking about that too, but I'm not sure it'd be useful yet, until we understand what ptloader is trying to do with it (or at least: not so useful as to be worth getting another team to make speculative prod changes -- maybe it would be a different equation if it were your own server). Like, even if it seems to solve the problem, do we trust it? Can we be sure that we haven't broken something else as a side effect, and just haven't noticed yet?

So far this could be a configuration problem, or it could be a bug in ptloader that's making it not work as intended despite being configured correctly, or it could be a fundamental architectural problem that makes Cyrus unable to work with this configuration. Or some combination thereof, I suppose. Still trying to figure out which! I'll post back once I've stepped through the source -- probably Friday or next week.

[elliefm]

Oh this is interesting. All those "ptload" lines in the log are coming from the auth_pts module, not the ptloader itself. The entry point appears to be "mycanonifyid", which is one of the callbacks we provide to SASL...

I think what's going on here is just that we've told SASL the authentication mechanism is PTS, so when it needs to authenticate to mailauth as mailproxy, it first asks PTS to canonify the name? So it doesn't seem to be ptloader doing something weird with SASL config that it shouldn't be looking at, but rather SASL doing whatever it was supposed to do (presumably)

I kinda think it might just be a matter of adding mailproxy to the other LDAP server, as we already speculated. But I'm reluctant to just recommend this, because I don't understand any of the pieces well enough to know whether it's a good idea or a terrible one!

It might be some kind of bug in the SASL ldapdb module, that the creds for talking to the LDAP server need to be canonified using the LDAP server? But once again, I don't know these pieces well enough... :(

[hagedose]

Oh this is interesting. All those "ptload" lines in the log are coming from the auth_pts module, not the ptloader itself. The entry point appears to be "mycanonifyid", which is one of the callbacks we provide to SASL...

That makes sense.

I think what's going on here is just that we've told SASL the authentication mechanism is PTS, so when it needs to authenticate to mailauth as mailproxy, it first asks PTS to canonify the name? So it doesn't seem to be ptloader doing something weird with SASL config that it shouldn't be looking at, but rather SASL doing whatever it was supposed to do (presumably)

I kinda think it might just be a matter of adding mailproxy to the other LDAP server, as we already speculated. But I'm reluctant to just recommend this, because I don't understand any of the pieces well enough to know whether it's a good idea or a terrible one!

I don't think it could hurt, though, so I will bring it up with the LDAP server team.

[hagedose]

Well, that was much simpler than I had expected, at least for a quick test. What's more: it worked!

canonified mailproxy -> mailproxy

With that ptload is happy, and the rest works as well.

[hagedose]

The issue itself is fixed, I would say, but I wonder if the documentation for this feature should be improved? Some of it belongs to cyrus-sasl, but at least the part regarding the interaction of ptloader and ldapdb should be documented in cyrus-imap, I think. So far the only places where ptloader is mentioned are the manpages for imapd.conf and for ptloader. Any suggestions where a proper configuration guide for the feature should be located? Perhaps here? https://www.cyrusimap.org/imap/concepts/deployment/authentication_and_authorization.html

[elliefm]

Reading that page, it feels like the eventual intent was to maybe have that page as a broad conceptual summary, and then maybe some subpages (that never got written yet) with more specific information for each of the things we support (LDAP, Kerberos, etc).

Would you be interested in writing up a page explaining how to configure LDAP in more detail? I think I would aim for something that assumes LDAP competence, and just explains how to bolt an existing LDAP infrastructure into Cyrus -- there are (I presume) already plenty of resources for how to manage one's particular choice of LDAP product. If you don't want to wrestle with the ReST doc format, I'm happy to take the content in some other format and handle the formatting conversion myself

This is one of the corners of Cyrus that Fastmail doesn't use, so we don't have expertise of our own to draw on when fleshing out the docs.

There might be useful old stuff to draw from in the legacy docs tree in git: https://github.com/cyrusimap/cyrus-imapd/tree/master/doc/legacy. Though I had a very quick skim just now and it doesn't seem to mention LDAP at all, so maybe not.

[hagedose]

FWIW, I'm planning on doing that when I find the time, but I don't see that happening until December.