SSSD / sssd

A daemon to manage identity, authentication and authorization for centrally-managed systems.
https://sssd.io
GNU General Public License v3.0
603 stars 247 forks source link

automount NSS lookup hangs while saving case insensitive cache key #7230

Open trispace opened 8 months ago

trispace commented 8 months ago

On a RHEL8 machine with sssd version 2.9.1-4.el8_9.5.x86_64 we are experiencing the following situation:

Accessing /tools/Mentor (where "Mentor" is the correct case sensitive directory name on the NFS server) the automount for the auto.tools map succeeds

[root@server~]# time ls /tools/Mentor >/dev/null

real    0m0.194s
user    0m0.001s
sys     0m0.003s

and the entry gets stored in the sssd cache:

[root@server ~]# ldbsearch -H cache_eda.ldb '(automountKey=Mentor)'
asq: Unable to register control with rootdse!
# record 1
dn: name=Mentor-rw\,intr\,hard\,retrans\3D20\,timeo\3D20      storage.example.com:/tools/&,name=auto.tools,cn=autofsmaps,cn=custom,cn=eda,cn=sysdb
automountInformation: -rw,intr,hard,retrans=20,timeo=20      storage.example.com
 :/tools/&
automountKey: Mentor
dataExpireTimestamp: 1709580330
name: Mentor-rw,intr,hard,retrans=20,timeo=20      storage.example.com:/tools/&
objectClass: automount
distinguishedName: name=Mentor-rw\,intr\,hard\,retrans\3D20\,timeo\3D20      s
 torage.example.com:/tools/&,name=auto.tools,cn=autofsmaps,cn=custom,cn=eda,cn=s
 ysdb

When accessing the nonexistent /tools/mentor automounted directory however, the access hangs for ~1,5 minutes:

[root@server ~]# time ls /tools/mentor >/dev/null
ls: cannot access '/tools/mentor': No such file or directory

real    1m38.298s
user    0m0.000s
sys     0m0.003s

while sssd keeps querying the LDAP server for the automount entry:

[04/Mar/2024:19:00:50.164270330 +0100] conn=19600 op=158 SRCH base="nisMapName=auto.tools,ou=Autofs,dc=eda,dc=example,dc=com" scope=2 filter="(&(cn=mentor)(objectClass=nisObject))" attrs="objectClass cn nisMapEntry"
[04/Mar/2024:19:00:50.164478747 +0100] conn=19600 op=158 RESULT err=0 tag=101 nentries=1 wtime=0.000123694 optime=0.000212338 etime=0.000334439
[04/Mar/2024:19:00:51.167036213 +0100] conn=19600 op=160 SRCH base="nisMapName=auto.tools,ou=Autofs,dc=eda,dc=example,dc=com" scope=2 filter="(&(cn=mentor)(objectClass=nisObject))" attrs="objectClass cn nisMapEntry"
[04/Mar/2024:19:00:51.167273333 +0100] conn=19600 op=160 RESULT err=0 tag=101 nentries=1 wtime=0.000135426 optime=0.000240499 etime=0.000355941
[04/Mar/2024:19:00:52.169674370 +0100] conn=19600 op=161 SRCH base="nisMapName=auto.tools,ou=Autofs,dc=eda,dc=example,dc=com" scope=2 filter="(&(cn=mentor)(objectClass=nisObject))" attrs="objectClass cn nisMapEntry"
[04/Mar/2024:19:00:52.170076060 +0100] conn=19600 op=161 RESULT err=0 tag=101 nentries=1 wtime=0.000195411 optime=0.000406834 etime=0.000598757
[...]
this search repeats until the ~1,5 minute timeout

and sssd logs:

(2024-03-04 19:00:49): [be[eda]] [sdap_autofs_save_entry] (0x0040): [RID#130] Cannot save autofs entry auto.tools:mentor [17]: File exists
   *  ... skipping repetitive backtrace ...
(2024-03-04 19:00:49): [be[eda]] [sbus_issue_request_done] (0x0040): sssd.DataProvider.Autofs.GetEntry: Error [17]: File exists
   *  ... skipping repetitive backtrace ...
(2024-03-04 19:00:50): [be[eda]] [sdap_autofs_save_entry] (0x0040): [RID#131] Cannot save autofs entry auto.tools:mentor [17]: File exists
   *  ... skipping repetitive backtrace ...
(2024-03-04 19:00:50): [be[eda]] [sbus_issue_request_done] (0x0040): sssd.DataProvider.Autofs.GetEntry: Error [17]: File exists
   *  ... skipping repetitive backtrace ...
(2024-03-04 19:00:51): [be[eda]] [sdap_autofs_save_entry] (0x0040): [RID#132] Cannot save autofs entry auto.tools:mentor [17]: File exists
   *  ... skipping repetitive backtrace ...
(2024-03-04 19:00:51): [be[eda]] [sbus_issue_request_done] (0x0040): sssd.DataProvider.Autofs.GetEntry: Error [17]: File exists
   *  ... skipping repetitive backtrace ...
(2024-03-04 19:00:52): [be[eda]] [sdap_autofs_save_entry] (0x0040): [RID#133] Cannot save autofs entry auto.tools:mentor [17]: File exists
   *  ... skipping repetitive backtrace ...
[...]
those lines repeat until the ~1,5 minute timeout

and autofs finally logs:

Mar 04 19:00:57 server.example.com automount[2390]: getautomntbyname: lookup(sss): getautomntbyname: Host is down

It seems that sssd tries to save 'auto.tools:mentor' in a loop but fails since 'auto.tools:Mentor' already exists.

Older sssd versions (such as the one in RHEL7) immediately return a "No such file or directory" while accessing the nonexistent /tools/mentor automounted directory.

While I think by default LDAP searches are case insensitive and thus a record is returned for both 'mentor' and 'Mentor' LDAP searches I'd like to know if sssd's cacheing behavior is correct in this case.

raven-au commented 7 months ago

Good catch and equally good report, thanks. The patch attached above fixes the problem.

I really don't know why glibc complains about this, there is no overflow. I think it's a bug in glibc.

I'll make an MR for the fix next week.

Oops! Didn't even realise, until now, that I posted this to the wrong place, sorry.