Limit the number of search results requested

[markcraig]

By default, the DS time-limit is 60 seconds and the size-limit is 1000 entries. 1000 entries in a UI with a default page size of 10 means 100 pages, more than a human would be expected to browse through.

Search should limit the number of results to avoid exceeding the size-limit setting. If the number of results exceeds the size-limit, display a message to the user indicating their search exceeded the size-limit. (For extra credit, show the estimated number of results.)

The search matching more than 1000 entries should not display results. Instead, the message should tell the user to refine their search to return fewer entries.

[markcraig]

There's something odd at least with the build of DS 7.6.0-SNAPSHOT from a week or so ago that I'm using. It doesn't seem to support simple paged results:

curl "http://localhost:3000/hdap/?_fields=cn,mail,manager,uid,uniqueMember&scope=sub&_queryFilter=cn+co+'test'+or+uid+eq+'test'+or+mail+co+'test'&_pageSize=1000"
{"code":400,"reason":"Bad Request","message":"Unavailable Critical Extension: La requête de recherche ne peut pas être exécutée car elle contient un contrôle critique avec l'identificateur d'objet 1.2.840.113556.1.4.319 qui n'est pas pris en charge par Directory Server pour ce type d'opérations"}

Maybe there's just something wrong with my build?

[markcraig]

I guess this doesn't work in LDAP either:

./opendj/bin/ldapsearch \
--port 1636 \
--hostname localhost \
--useSSL \
--trustAll \
--baseDn "" \
--simplePageSize 1000 \
"(|(cn=*test*)(uid=test)(mail=*test*))" 1.1

# The LDAP search request failed: 12 (Unavailable Critical Extension)
# Additional Information:  The search request cannot be processed because it contains a critical control with OID 1.2.840.113556.1.4.319 that is not supported by the Directory Server for this type of operation

It does however work for a base DN where the data is:

./opendj/bin/ldapsearch \
--port 1636 \
--hostname localhost \
--useSSL \
--trustAll \
--baseDn "dc=example,dc=com" \
--simplePageSize 1000 \
"(|(cn=*test*)(uid=test)(mail=*test*))" 1.1

dn: uid=user.3466,ou=People,dc=example,dc=com

dn: uid=user.7168,ou=People,dc=example,dc=com
...

[markcraig]

The present implementation for a basic search uses "" as the base DN (search under /hdap).

Workarounds:

• For the demo, search under /hdap/dc=com/dc=example by default. This could rely on the accident that the other public naming contexts in the DS eval profile come later alphabetically (uid=admin, uid=monitor), but it feels pretty unsatisfying, even if it's what you'd do fairly naturally for drop-down list in an "advanced search" UI.

• HDAP returns 413 when a search exceeds the size limit:

  curl --cacert ca-cert.pem --user uid=monitor:password "https://localhost:8443/hdap/?_fields=cn,mail,manager,uid,uniqueMember&scope=sub&_queryFilter=mail+co+'user.12'" 
  {"code":413,"reason":"Request Entity Too Large","message":"Size Limit Exceeded: This search operation has sent the maximum of 1000 entries to the client"}

  In this case, indicate the user needs to refine their search.

[matthew-swift]

As you have noticed, CREST doesn't appear to support server size limits: it performs the search, consumes the 1000 results then gets the size limit error. At this point CREST discards the 1000 search results and just returns the HTTP error. For the UI this behavior is ok because you don't want to display any results anyway and instead just show a "refine your query" message.

Simple paged results has the advantage that it sometimes includes a total result estimate which could be helpful feedback to the end-user. However, as you spotted, paged results doesn't work across backends.