Getting an LDAPException(resultCode=4 (size limit exceeded) even though I am using paging with correct number of records?

[nddipiazza]

I am having the issue described here: https://github.com/pingidentity/ldapsdk/issues/65

So as suggested I am using a paged control:

    ASN1OctetString resumeCookie = null;
    do {
      searchRequest.setControls(new SimplePagedResultsControl(500, null));
      SearchResult searchResult = search(searchRequest);
   ....

And you can see the paged controls are there, and my sizeLimit = 0.

Yet I get this error:

java.lang.Exception: Request=SearchRequest(baseDN='DC=example,DC=com', scope=SUB, deref=NEVER, sizeLimit=0, timeLimit=0, filter='(&(whenChanged>=20210502222706.0Z)(|(&(objectclass=user)(sAMAccountName=*))(&(objectclass=group))))', attrs={whenChanged, telephoneNumber, manager, sAMAccountName, objectClass, member, distinguishedName, objectSid, memberOf, userPrincipalName}, controls={SimplePagedResultsControl(pageSize=500, isCritical=false)}), Error=LDAPException(resultCode=4 (size limit exceeded), numEntries=1000, numReferences=1, errorMessage='size limit exceeded', ldapSDKVersion=4.0.9, revision=29290')

Is there some reason you will get size limit exceeded even though I am 1) using paging, and 2) have sizeLimit=0 ?

Is there some reason it's trying to return more records than allowed in my paging control?

[dirmgr]

First, setting a size limit of zero in the search request only sets the requested size limit. Even if the client requests no size limit, the server can choose to impose a limit anyway,and most do, especially for non-administrative accounts. In this case, it sounds like the server is imposing a maximum size limit of 1000 entries.

Second, the fact that you're getting back more than 500 entries in the request makes it sound like the simple paged results request control is not being honored. The constructor that you're using will result in the control having a criticality of false, which means that it's acceptable for the server to ignore that control if it doesn't support it or can't honor it for that request for some reason. If you create the control with a criticality of true (using new SimplePagedResults(500, null, true)), then the server must either honor the control or reject the request with a result code of UNAVAILABLE_CRITICAL_EXTENSION (12).

It may be that the server you're using either doesn't support the simple paged results control, or that it doesn't support it in the way that you're using it. From some of the attributes that you seem to be using in the search request, it sounds like you're probably using Active Directory, and since Microsoft is one of the organizations involved in creating RFC 2696 that defines the simple paged results control, I assume that Active Directory should support it. However, it also looks like you might be using a base DN of "dc=example,dc=com", and I think that Active Directory also likes using multiple naming contexts, including those further down the tree. It's possible that it doesn't support paged searches that span multiple naming contexts. I'm definitely not an Active Directory expert so I can't help you with that part of it, but you can at least try making the control critical and seeing if that causes the server to either honor the control or reject the request.

[darioseidl]

I was facing the same situation with an OpenLDAP server. The server accepts the pagination request and returns pages, but after several pages still throws a "size limit exceeded" LDAPException.

It looks like OpenLDAP servers have a SizeLimit that applies to the total number of entries, not the number of entries in a page (that would be MaxPageSize, and would throw a different exception when exceeded).

The OpenLDAP docs says about the size limit:

If the LDAP client adds the pagedResultsControl to the search operation, the hard size limit is used by default, because the request for a specific page size is considered an explicit request for a limitation on the number of entries to be returned. However, the size limit applies to the total count of entries returned within the search, and not to a single page.

You can try increasing the sizeLimit in the SearchRequest constructor, that's a different thing than pagination and could (depending on the server) override the server size limit, but it didn't work in my case. In that case, it can probably only be increased on the server.