Unindexed filter on description attribute

[SilleBille]

This issue was migrated from Pagure Issue #3083.Originally filed by cheimes on 2018-12-04

After a user has filed a FreeIPA bug report about slow LDAP queries, I started to look into query performance again. 389-DS complains about one unindexed filter on description regularly. It seems to be related to Dogtag. I think the slow query occurs when IPA uses its RA agent certificate to log into Dogtag's admin interface:

  Unindexed Component #275 (notes=U)                                                                                                                                                                                                
-  Date/Time:             03/Dec/2018:16:14:47                                                                                                                                                                                    
-  Connection Number:     149
-  Operation Number:      998
-  Etime:                 0.0053610845
-  Nentries:              1
-  IP Address:            10.37.170.201
-  Search Base:           ou=people,o=ipaca
-  Search Scope:          2 (subtree)
-  Search Filter:         (description=2;7;cn=certificate authority,o=ipa.example;cn=ipa ra,o=ipa.example)
-  Bind DN:               cn=directory manager

To fix the issue either Dogtag or FreeIPA should create an eq index on description. I'm reporting the issue here because I don't know if the problem also affects non-IPA uses of Dogtag.

Also see https://pagure.io/dogtagpki/issue/2603

[SilleBille]

Posted by cheimes on 2018-12-04:

Update: I don't fully understand why 389-DS considers a search for description in ou=people,o=ipaca as unindexed filter. Dogtag already creates an index for its database:

dn: cn=description,cn=index,cn=ipaca,cn=ldbm database,cn=plugins,cn=config
cn: description
nsIndexType: eq
nsIndexType: pres
nsSystemIndex: false
objectClass: top
objectClass: nsIndex

The problem might be caused by a missing index task. I see cn=index1160589769, cn=index, cn=tasks, cn=config from ./base/ca/shared/conf/vlvtasks.ldif and cn=index1160527115,cn=index,cn=tasks,cn=config from ./base/kra/shared/conf/vlvtasks.ldif in the access log of 389-DS. There are no entries for the index tasks index1160589770 and index1160589771 in 389-DS' access log.

[SilleBille]

Posted by cheimes on 2018-12-04:

Debug logs for CA and KRA don't show the index tasks on the first IPA master. On the replica, there is only an index task for index1160589770 (that's CA's indextasks.ldif) but not for KRA indextasks.ldif.

master

# grep -R index11 /var/log/pki/
/var/log/pki/pki-tomcat/ca/debug.2018-12-03.log:2018-12-03 16:12:40 [https-jsse-nio-8443-exec-10] FINE: Checking wait_dn cn=index1160589769, cn=index, cn=tasks, cn=config
/var/log/pki/pki-tomcat/kra/debug.2018-12-03.log:2018-12-03 16:18:40 [https-jsse-nio-8443-exec-10] FINE: Checking wait_dn cn=index1160527115, cn=index, cn=tasks, cn=config

replica

grep -R index11 /var/log/pki/
/var/log/pki/pki-tomcat/ca/debug.2018-12-03.log:2018-12-03 16:41:24 [https-jsse-nio-8443-exec-10] FINE: Checking wait_dn cn=index1160589770,cn=index,cn=tasks,cn=config
/var/log/pki/pki-tomcat/ca/debug.2018-12-03.log:2018-12-03 16:41:27 [https-jsse-nio-8443-exec-10] FINE: Checking wait_dn cn=index1160589769, cn=index, cn=tasks, cn=config
/var/log/pki/pki-tomcat/kra/debug.2018-12-03.log:2018-12-03 16:47:10 [https-jsse-nio-8443-exec-10] FINE: Checking wait_dn cn=index1160527115, cn=index, cn=tasks, cn=config

[SilleBille]

Posted by cheimes on 2018-12-04:

The index task files are not installed on master and only partly installed on the replica.

master

# find /etc/pki/pki-tomcat/ -name indextasks.ldif
# rpm -qf /usr/share/pki/ca/conf/indextasks.ldif /usr/share/pki/kra/conf/indextasks.ldif
pki-ca-10.6.8-1.fc29.noarch
pki-kra-10.6.8-1.fc29.noarch

replica

# find /etc/pki/pki-tomcat/ -name indextasks.ldif
/etc/pki/pki-tomcat/ca/indextasks.ldif