Handle large find all requests [DATALDAP-138]

[spring-projects-issues]

Jake Zarobsky opened DATALDAP-138 and commented

When querying ADLDS, there is a default limit to the number of objects returned from a query. In order to improve Spring Data LDAP to be more consistent with other repositories that developers interact with, we should update the findAll method to use a PagesResultsDirContextProcessor to essentially page through objects returned from the LDAP provider.

I've provided an example below on how we query LDAP since the findAll method isn't suiting our needs in its current state.

private <T> Stream<T> getAllObjectsForClass(final Class<T> clazz, final Filter baseFilter) { 
final PagedResultsDirContextProcessor processor = new PagedResultsDirContextProcessor(500); 
final SearchControls searchControls = new SearchControls(); searchControls.setSearchScope(SearchControls.SUBTREE_SCOPE); final ObjectDirectoryMapper objectDirectoryMapper = ldapTemplate.getObjectDirectoryMapper(); 
final String filter = objectDirectoryMapper.filterFor(clazz, baseFilter).encode(); final ContextMapper<T> contextMapper = ctx -> objectDirectoryMapper.mapFromLdapDataEntry((DirContextOperations) ctx, clazz);
return SingleContextSource.doWithSingleContext(ldapTemplate.getContextSource(), operations -> { 
final List<T> results = new LinkedList<>(); 
do { 
results.addAll(operations.search("", filter, searchControls, contextMapper, processor)); 
} while (processor.hasMore()); 
return results; }).stream(); }

---

Issue Links:

• DATALDAP-30 Paging LDAP query ("duplicates")

[spring-projects-issues]

Mark Paluch commented

Thanks for your suggestion. We had a similar request with DATALDAP-30. We didn't add pagination support as Spring LDAP can be configured with a ContextSource that uses a different connection for each operation. The handling of the pagination token isn't defined across multiple connections and introducing pagination would use non-portable behavior that may or may not work, depending on the LDAP server implementation.

We suggest overriding the methods that require customization on your end with the functionality that is required for your arrangement