LDAP group search doesn't work

[captnbp]

Describe the bug LDAP group search doesn't work:

WARNING! The following warnings were returned from Vault:

  * no LDAP groups found in groupDN 'ou=groups,dc=planetexpress,dc=com'; only
  policies from locally-defined groups available

The UserDN variable seems to be empty when rendered in the LDAP groupfilter

To Reproduce

Steps to reproduce the behavior:

1. Run docker run --privileged -d -p 389:389 --name ldap rroemhild/test-openldap to start a provisioned LDAP server
2. Run vault write auth/ldap/config url="ldap://<YOUR LDAP HOST IP>:389" userdn="ou=people,dc=planetexpress,dc=com" groupdn="ou=people,dc=planetexpress,dc=com" groupattr="cn" upndomain="planetexpress.com" insecure_tls=true userattr=uid starttls=false binddn="cn=admin,dc=planetexpress,dc=com" bindpass='GoodNewsEveryone' to setup LDAP auth
3. Tail LDAP server logs docker logs -f ldap
4. Run vault write auth/ldap/groups/admin_staff policies=default
5. Login vault login -method=ldap username=hermes (Password: hermes)
6. See error:

   
   Password (will be hidden): 
   Success! You are now authenticated. The token information displayed below
   is already stored in the token helper. You do NOT need to run "vault login"
   again. Future Vault requests will automatically use this token.

WARNING! The following warnings were returned from Vault:

• no LDAP groups found in groupDN 'ou=people,dc=planetexpress,dc=com'; only policies from locally-defined groups available

Key Value

---

token s.96pPWGAjcVKcLcpnHhaWUAJV token_accessor PJmL0AUsXhrVGA59PJK18rhi token_duration 768h token_renewable true token_policies ["default"] identity_policies [] policies ["default"] token_meta_username hermes

5c7ad477 conn=1049 fd=18 ACCEPT from IP=192.168.1.45:35202 (IP=0.0.0.0:389) 5c7ad477 conn=1049 op=0 BIND dn="cn=admin,dc=planetexpress,dc=com" method=128 5c7ad477 conn=1049 op=0 BIND dn="cn=admin,dc=planetexpress,dc=com" mech=SIMPLE ssf=0 5c7ad477 conn=1049 op=0 RESULT tag=97 err=0 text= 5c7ad477 conn=1049 op=1 SRCH base="ou=people,dc=planetexpress,dc=com" scope=2 deref=0 filter="(uid=hermes)" 5c7ad477 conn=1049 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text= 5c7ad477 conn=1049 op=2 BIND anonymous mech=implicit ssf=0 5c7ad477 conn=1049 op=2 BIND dn="cn=Hermes Conrad,ou=people,dc=planetexpress,dc=com" method=128 5c7ad477 conn=1049 op=2 BIND dn="cn=Hermes Conrad,ou=people,dc=planetexpress,dc=com" mech=SIMPLE ssf=0 5c7ad477 conn=1049 op=2 RESULT tag=97 err=0 text= 5c7ad477 conn=1049 op=3 BIND anonymous mech=implicit ssf=0 5c7ad477 conn=1049 op=3 BIND dn="cn=admin,dc=planetexpress,dc=com" method=128 5c7ad477 conn=1049 op=3 BIND dn="cn=admin,dc=planetexpress,dc=com" mech=SIMPLE ssf=0 5c7ad477 conn=1049 op=3 RESULT tag=97 err=0 text= 5c7ad477 conn=1049 op=4 SRCH base="ou=people,dc=planetexpress,dc=com" scope=2 deref=0 filter="(?userPrincipalName=cn=Hermes Conrad,ou=people,dc=planetexpress,dc=com)" 5c7ad477 conn=1049 op=4 SEARCH RESULT tag=101 err=0 nentries=0 text= 5c7ad477 conn=1049 op=5 SRCH base="ou=people,dc=planetexpress,dc=com" scope=2 deref=0 filter="(|(memberUid=hermes)(member=)(uniqueMember=))" 5c7ad477 conn=1049 op=5 SRCH attr=cn 5c7ad477 <= bdb_equality_candidates: (uniqueMember) not indexed 5c7ad477 conn=1049 op=5 SEARCH RESULT tag=101 err=0 nentries=0 text= 5c7ad477 conn=1049 fd=18 closed (connection lost)

We can see that `5c7ad477 conn=1049 op=5 SRCH base="ou=people,dc=planetexpress,dc=com" scope=2 deref=0 filter="(|(memberUid=hermes)(member=)(uniqueMember=))"` which is rendered by `(|(memberUid={{.Username}})(member={{.UserDN}})(uniqueMember={{.UserDN}}))` is empty when the `{{.UserDN}}` template should contain the user DN fetched in the previous queries.

**Expected behavior**
Vault should render `{{.UserDN}}` with the user DN fetched in the previous LDAP queries and map the LDAP group to the policy:

vault login -method=ldap username=hermes Password (will be hidden): Success! You are now authenticated. The token information displayed below is already stored in the token helper. You do NOT need to run "vault login" again. Future Vault requests will automatically use this token.

Key Value

---

token s.sE3eGx1vtectpm1sabQX2TWs token_accessor 1r2Wq8l3m4OAzkyXR6XQni3m token_duration 768h token_renewable true token_policies ["default"] identity_policies [] policies ["default"] token_meta_username hermes

5c7ad173 conn=1040 op=5 SRCH base="ou=groups,dc=planetexpress,dc=com" scope=2 deref=0 filter="(|(memberUid=hermes)(member=cn=hermes conrad,ou=people,dc=planetexpress,dc=com)(uniqueMember=cn=hermes conrad,ou=people,dc=planetexpress,dc=com))" 5c7ad173 conn=1040 op=5 SRCH attr=cn 5c7ad173 conn=1040 op=5 SEARCH RESULT tag=101 err=0 nentries=1 text=

I also tried with the following group filter: `(&(objectClass=group)(member={{.UserDN}}))` -> Same issue

**Environment:**
* Vault Server Version (retrieve with `vault status`): 

Key Value

---

Seal Type shamir Initialized true Sealed false Total Shares 1 Threshold 1 Version 1.0.3 Cluster Name vault-cluster-64f5aef4 Cluster ID 70075259-294b-5190-5b0b-c149453b521a HA Enabled false

* Vault CLI Version (retrieve with `vault version`): `v1.0.3 ('85909e3373aa743c34a6a0ab59131f61fd9e8e43')`
* Server Operating System/Architecture: `Ubuntu 18.04.2 x86_64`

Vault server configuration file(s):

```hcl
storage "file" {
  path = "/var/lib/vault/data"
}
listener "tcp" {
  address = "127.0.0.1:8200"
  tls_disable = true
}
api_addr = "http://127.0.0.1:8200"

Additional context

[captnbp]

I removed upndomain="planetexpress.com" from

vault write auth/ldap/config url="ldap://<YOUR LDAP HOST IP>:389" userdn="ou=people,dc=planetexpress,dc=com" groupdn="ou=people,dc=planetexpress,dc=com" groupattr="cn" upndomain="planetexpress.com" insecure_tls=true userattr=uid starttls=false binddn="cn=admin,dc=planetexpress,dc=com" bindpass='GoodNewsEveryone'

and now it works fine.

I don't know if it is a bug or just a problem between the chair and the keyboard :-/

[NeilSorensen]

Definitely a bug. In GetUserDN (https://github.com/hashicorp/vault/blob/master/helper/ldaputil/client.go#L149), if the bindDN is actually a distinguished name (because binddn and bindpass are set) and upndomain is also set, it will try to look up users with a (userPrincipalName=cn=hermes,ou=people,dc=planetexpress,dc=com) filter.

It seems like GetUserBindDN and GetUserDN should be using the same logic there.

[jefferai]

@NeilSorensen I'm not following your comment. Which part of the logic do you think should change? It seems like the? a? bug may be that line 134 should be moved up as the first if condition. I'm not sure if that's related to this problem or not though, mostly because I don't know in the GetUserDN function what the logic should be.

[NeilSorensen]

GetUserBindDN (https://github.com/hashicorp/vault/blob/master/helper/ldaputil/client.go#L101) returns three different things, depending on the configuration:

1) If DiscoverDN is true, or binddn and bindpass are set, it returns the distinguished name of the user, after looking it up.(e.g. cn=hermes,ou=people,dc=planetexpress,dc=com) 2) If neither of the above is true, and upndomain is set, it returns the user email address (e.g. hermes@planetexpress.com) 3) An attempt to build the distinguished name without contacting vault (from the example above, the constructed DN would be uid=hermes,ou=people,cd=planetexpress,dc=com)

The result of GetUserBindDN is passed to GetUserDN as bindDN before Vault attempts to look up groups (https://github.com/hashicorp/vault/blob/master/builtin/credential/ldap/backend.go#L130)

If upndomain is set, GetUserDN assumes that bindDN is a user email, and tries to look up the DN by executing a ldap query for (userPrincipalName={{bindDN}}) (https://github.com/hashicorp/vault/blob/master/helper/ldaputil/client.go#L151). It then returns the DN of the last user that this search returns (also probably bad, but maybe you can get away with it...)

The bug happens because if either DiscoverDN is true (probably not likely when upndomain is set, but possible) or binddn and bindpass are set, bindDN is not a user email, but is instead the full DN of the user. So you end up searching for users with (userPrincipalName=cn=hermes,ou=people,dc=planetexpress,dc=com), and when that returns no users, you set the userDN as empty string when searching for groups

[jefferai]

Thanks for the explanation! Do you agree that moving the condition on line 134 first (so that if upndomain is set we use that, then do the discoverdn/binddn logic, then the fallback) would fix the issue?

[NeilSorensen]

That might make group discovery work, but at the cost of potentially breaking individual authentication. If it were me, I'd change GetUserDN to use the same logic as GetUserBindDN

[jefferai]

Why do you think that would break individual auth? That comment suggests that you think that honoring upndomain, if it's set, will break auth -- but shouldn't we always be honoring that if it's set?

[mrmodolo]

I have the same error! However, if I remove the 'User Principal (UPN) Domain' I can no longer authenticate.

With the 'User Principal (UPN) Domain' attribute authentication happens normally!

vault login -method=ldap username=modolo
Password (will be hidden): 
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

WARNING! The following warnings were returned from Vault:

  * no LDAP groups found in groupDN
  'CN=AdmLinux,OU=Grupos,DC=globosat,DC=net,DC=br'; only policies from
  locally-defined groups available

Key                    Value
---                    -----
token                  xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
token_accessor         yyyyyyyyyyyyyyyyyyyyyyyyy
token_duration         10h
token_renewable        true
token_policies         ["default"]
identity_policies      []
policies               ["default"]
token_meta_username    modolo

[mrmodolo]

Hi!

It may be that my settings help someone else! In my case, the internal domain used in my company is different from the email domain!

Internal domain: mydomain.net.br Domain e-mail: mydomain.com.br

"Port 3268. This port is used for queries specifically targeted for the global catalog. LDAP requests sent to port 3268 can be used to search for objects in the entire forest. However, only the attributes marked for replication to the global catalog can be returned. For example, a user’s department could not be returned using port 3268 since this attribute is not replicated to the global catalog."

I was able to get the correct groups using the settings below.

URL 
ldap://dc01.mydomain.net.br:3268, ldap://dc02.mydomain.net.br:3268

User Attribute: samaccountname
User Principal (UPN) Domain: mydomain.net.br

Name of Object to bind (binddn): CN=vault,OU=Users,OU=ION,DC=mydomain,DC=net,DC=br
User DN: DC=mydomain,DC=net,DC=br
Bindpass : xxxxxxxx

Groupfilter: (&(objectClass=person)(mail={{.Username}}@mydomain.com.br))
Group Attribute: memberOf
Group DN: DC=globosat,DC=net,DC=br

I do not know if this is the best way, but I hope this helps!

[pouyana]

Hi

from the v.1.1.1 I have the same Problem. The old configuration that retrieved the groups does not work any more. I am able to login, but the groups are not found. The only change to LDAP helper from here https://github.com/hashicorp/vault/compare/v1.1.0...v1.1.1 was 8ade9c18ef9646494566598c850d57c301e5f729.

My configuration for help:

URL: ldaps://my-ldap-server:636
Userattr: uid
UPN: empty

Bind: empty

group: (&(objectClass=person)(uid={{.Username}}))
group attr: groupMembership
dn: ou=accounts,o=my-ldap-org

UPDATED

From the v.1.1.1 the groups are parsed and their CN is used for the group policy comparison. This make the old group policies that contain the DNs not work properly.

Before v.1.1.1

vault list auth/ldap/groups
Keys
----
cn=group-a,ou=groups,o=my-org
cn=group-b,ou=groups,o=my-org

After v.1.1.1

vault list auth/ldap/groups
Keys
----
group-a
group-b

Is it the intended consequence of the given change?

[sparkacus]

I've also run into this issue and I can't quite work it out - Unfortunately my LDAP skillz aren't amazing.

I use JumpCloud.

I enabled vault debug logging and as far as I can tell (maybe I'm completely wrong) it's connecting with the wrong binddn... it's using the user I try to authenticate with rather than what's defined in the ldap config e.g.

Jul 18 18:55:54 ip-172-16-2-0 vault[23217]: 2019-07-18T18:55:54.484Z [DEBUG] auth.ldap.auth_ldap_0b1785b7: user binddn fetched: username=myusername binddn=uid=myusername,ou=Users,o=<ORGID>,dc=jumpcloud,dc=com
Jul 18 18:55:54 ip-172-16-2-0 vault[23217]: 2019-07-18T18:55:54.512Z [DEBUG] auth.ldap.auth_ldap_0b1785b7: compiling group filter: group_filter=(|(memberUid={{.Username}})(member={{.UserDN}})(uniqueMember={{.UserDN}}))
Jul 18 18:55:54 ip-172-16-2-0 vault[23217]: 2019-07-18T18:55:54.512Z [DEBUG] auth.ldap.auth_ldap_0b1785b7: searching: groupdn=ou=Users,o=<ORGID>,dc=jumpcloud,dc=com rendered_query=(|(memberUid=myusername)(member=uid=myusername,ou=Users,o=<ORGID>,dc=jumpcloud,dc=com)(uniqueMember=uid=myusername,ou=Users,o=<ORGID>,dc=jumpcloud,dc=com))
Jul 18 18:55:54 ip-172-16-2-0 vault[23217]: 2019-07-18T18:55:54.539Z [DEBUG] auth.ldap.auth_ldap_0b1785b7: groups fetched from server: num_server_groups=0 server_groups=[]
`

[balan2010]

Not working in v1.2.2 as well

WARNING! The following warnings were returned from Vault:

• no LDAP groups found in groupDN 'ou=groups,dc=company,dc=com'; only policies from locally-defined groups available

  auth.ldap.auth_ldap_8a4e4bdf: groups fetched from server: num_server_groups=0 server_groups=[]

[jefferai]

Note: I hid a comment above because it's actually a totally separate issue from this one, and has already been resolved.

[rnsc]

Is there any fix planned for this?

[dihalk]

Not working in v1.2.2 as well

WARNING! The following warnings were returned from Vault:

• no LDAP groups found in groupDN 'ou=groups,dc=company,dc=com'; only policies from locally-defined groups available

auth.ldap.auth_ldap_8a4e4bdf: groups fetched from server: num_server_groups=0 server_groups=[]

same error . not working with version 1.2.3 as well

[devops-rob]

I'm also getting the same error in version 1.2.3. It's definitely a blocker for us here. Glancing through the comments, i agree with @jefferai re moving the updomain condition up as it should always honour that if set (in theory).

[dunkelbunt1]

Have the same error in version 1.2.2 and 1.2.3. The error doesn't occur in version 1.2.1

As a workaround you can remove the upndomain or leave it blank.

[sdeoras]

leaving the upndomain blank worked for me as well.

[devops-rob]

hmmm...i'm missing something then because leaving the upndomain blank didn't work for me. I removed it completely and it wouldn't let me log in at all. Are any of you able to share your configuration (with sensitive values omitted of course) as a pointer please @sdeoras @dunkelbunt1 ?

[dunkelbunt1]

@devops-rob have a look. I am using Terraform to configure Vault. See below for an example.

resource "vault_ldap_auth_backend" "ldap" { path = "ldap" url = "ldap://domain.com" starttls = false insecure_tls = false discoverdn = false deny_null_bind = true userattr = "sAMAccountName" binddn = var.binddn userdn = "OU=User,OU=Usergroup,DC=domain,DC=com" bindpass = var.bindpass groupfilter = "(&(objectClass=group)(member:1.2.840.113556.1.4.1941:={{.UserDN}}))" groupattr = "memberOf" groupdn = "OU=Security,OU=Manged Users,OU=Location,OU=Usergroup,DC=Domain,DC=Com" use_token_groups = true case_sensitive_names = true }

[ilatvala]

I have been using vault 1.1.3 with LDAP auth and it has worked ok. Was planning to try raft storage backend and wanted to try vault 1.3, but now I get also this "no LDAP groups found in groupDN". Tried with empty upndomain, but that did not help. Any pointers how to troubleshoot this further, or is this really a bug that has appeared after 1.1.3?

[ilatvala]

Got it to work now in vault 1.3 with a new groupfilter... groupfilter="(&(uid={{.Username}})(MemberOf=*))"

[Jamby93]

I solved with a mix of the above configuration:

                userdn: "OU=User,DC=contoso,DC=com"
                binddn: "CN=vault,OU=Security,DC=contoso,DC=com"
                bindpassword: "<REDACTED>"
                groupdn: "OU=User,DC=contoso,DC=com"
                groupfilter: "(&(objectClass=person)(samAccountName={{.Username}}))"
                groupattr: memberOf
                use_token_groups: false
                userattr: samAccountName

Using token groups in our case has the strange effect of getting the groups of the binddn user (vault) instead of the logging-in one. I struggle to understand how 'memberOf' groupattr with a groupfilter that resemble a user search can even work. I had little to no problem with Vault 1.1.X

[jmturwy]

@sparkacus

I've also run into this issue and I can't quite work it out - Unfortunately my LDAP skillz aren't amazing.

I use JumpCloud.

I enabled vault debug logging and as far as I can tell (maybe I'm completely wrong) it's connecting with the wrong binddn... it's using the user I try to authenticate with rather than what's defined in the ldap config e.g.

Jul 18 18:55:54 ip-172-16-2-0 vault[23217]: 2019-07-18T18:55:54.484Z [DEBUG] auth.ldap.auth_ldap_0b1785b7: user binddn fetched: username=myusername binddn=uid=myusername,ou=Users,o=<ORGID>,dc=jumpcloud,dc=com
Jul 18 18:55:54 ip-172-16-2-0 vault[23217]: 2019-07-18T18:55:54.512Z [DEBUG] auth.ldap.auth_ldap_0b1785b7: compiling group filter: group_filter=(|(memberUid={{.Username}})(member={{.UserDN}})(uniqueMember={{.UserDN}}))
Jul 18 18:55:54 ip-172-16-2-0 vault[23217]: 2019-07-18T18:55:54.512Z [DEBUG] auth.ldap.auth_ldap_0b1785b7: searching: groupdn=ou=Users,o=<ORGID>,dc=jumpcloud,dc=com rendered_query=(|(memberUid=myusername)(member=uid=myusername,ou=Users,o=<ORGID>,dc=jumpcloud,dc=com)(uniqueMember=uid=myusername,ou=Users,o=<ORGID>,dc=jumpcloud,dc=com))
Jul 18 18:55:54 ip-172-16-2-0 vault[23217]: 2019-07-18T18:55:54.539Z [DEBUG] auth.ldap.auth_ldap_0b1785b7: groups fetched from server: num_server_groups=0 server_groups=[]
`

Not sure if you ever got this to work with Jumpcloud but i got it working on 1.3.4 vault:

`Key Value

---

binddn uid=User,ou=Users,o=,dc=jumpcloud,dc=com case_sensitive_names false certificate n/a deny_null_bind true discoverdn false groupattr n/a groupdn ou=Users,o=,dc=jumpcloud,dc=com groupfilter (&(objectClass=groupOfNames)(cn=*)) insecure_tls false starttls false tls_max_version tls12 tls_min_version tls12 token_bound_cidrs [] token_explicit_max_ttl 0s token_max_ttl 0s token_no_default_policy false token_num_uses 0 token_period 0s token_policies [] token_ttl 0s token_type default upndomain n/a url ldaps://ldap.jumpcloud.com use_pre111_group_cn_behavior false use_token_groups false userattr uid userdn ou=Users,o=,dc=jumpcloud,dc=com`

[pratiklotia]

I'm using 1.3.2+ent and got the same error.

Took @Jamby93 's suggestion and changed the group filter and group dn. It worked fine after that - no ldap errors. Thank you.

[StevePerezUK]

Got it to work now in vault 1.3 with a new groupfilter... groupfilter="(&(uid={{.Username}})(MemberOf=*))"

That worked for me. Redhat Identity Manager. Thanks