search

RocketChat / Rocket.Chat

The communications platform that puts data protection first.
https://rocket.chat/
Other
40.51k stars 10.55k forks source link

LDAP Sync: final batch not processed (without pagination) #9279

Closed sscholl closed 6 years ago

sscholl commented 6 years ago

Description:

I use LDAP without pagination (by setting LDAPP_Search_Page_Size to 0), because my LDAP Server is not able to provide pragination. When I use the LDAP Sync Feature (by hand or by cronjob), 500 users are imported at maximum. I guess this is because of the missing pagination feature, but my LDAP sends all ~800 users.

Can you increase the default value or make it configurable?

Thank you!!!

Server Setup Information:

Steps to Reproduce:

  1. Configure LDAP with more than 500 users
  2. set LDAPP_Search_Page_Size to 0
  3. set Search_Size_Limit to 1000
  4. Execute LDAP_Sync_Now

Expected behavior:

All LDAP Users shoud be imported (more than 500).

Actual behavior:

Only 500 users are imported. After log message LDAPSync ➔ info Import running. Users imported until now: 500 import stops.

Relevant logs:

20171229-11:58:44.846(0) rocketchat_logger rocketchat_logger.js:278 LDAPSync ➔ info Import running. Users imported until now: 500

rodrigok commented 6 years ago

@sscholl There is no hard limit in our code.

Was it working before version 0.60? Could it be a hard search limit in your LDAP server configuration?

rodrigok commented 6 years ago

We tested imports with more then 200.000 users with and without pagination.

sscholl commented 6 years ago

@rodrigok Thanks for your answer! I can see, that my LDAP is returning ~800 users, so my guess here is Rocket.Chat. I see the value 500 for example here in the source code: https://github.com/RocketChat/Rocket.Chat/blob/e2c0fc958c85c796a8e111b3eb060645336bfcde/packages/rocketchat-ldap/server/ldap.js#L389

Or is it possible, that I have a problem with asynchronous jobs?

EDIT: I did not test it before 0.60. Thats my first time. EDIT2: I was able to import the users by using setting LDAP_Search_Page_Size to 10000 and Search_Size_Limit 100000. But this gave me the missing pagination error again. However initially the users are present now. Further automatic Full-Syncs still not possible - of course, I disabled pagination again.

PagedError: missing paged control
  at SearchPager._onEnd (/app/bundle/programs/server/npm/node_modules/ldapjs/lib/client/search_pager.js:91:15)
rodrigok commented 6 years ago

@sscholl That's the internal batch control, is not related with LDAP pagination.

But I found the error, it's not processing the final results of the final batch, I'll fix ASAP, release a new version and let you know.

Thanks

sscholl commented 6 years ago

Very nice!!! Thank you :)