[Question] Importing LDAP Objects - not importing Group members if there are more than 1500 members

[Pixa1]

Environment:

• LDAP Server Type: ActiveDirectory
• LdapRecord-Laravel Major Version: v3
• PHP Version: 8.3

Describe the bug: I'm using synchronizer to import AD objects to the database (example from here https://ldaprecord.com/docs/laravel/v3/importing#introduction)

This is the attribute handler class:

    public function handle(LdapGroup $ldap, DatabaseGroup $database)
    {
        $database->name = $ldap->getFirstAttribute('cn');
        $database->email = $ldap->getFirstAttribute('mail');
        $database->members = $ldap->member;
        $database->managedby = $ldap->getFirstAttribute('managedby');
        $database->msexchcomanagedbylink = $ldap->getAttribute('msexchcomanagedbylink');
        $database->managedobjects = $ldap->getAttribute('managedobjects');
        $database->distinguishedname = $ldap->getFirstAttribute('distinguishedname');
    }

This is the part of the code to sync:

        $config = ['sync_attributes' => \App\Ldap\GroupAttributeHandler::class];
        $synchronizer = new Synchronizer(EloquentGroup::class, $config);
         foreach (LdapGroup::all() as $group) {
            $synchronizer->run($group)->save();
        }; 

The group is synced to the database, but the members list is empty. I noticed when I query the specific group that contains more than 1500 members, the members attribute is different, it looks like this:

...
 "member" => [],
 "member;range=0-1499" => [ .... ],
...

Is there any way to get around this?

[stevebauman]

Hi @Pixa1,

You will need to use the model's relationship method instead of its attribute:

foreach (LdapGroup::all() as $group) {
    $members = [];

    $group->members()->each(function (LdapGroup $group) use (&$members) {
        $members[] = $group->getDn();
    });

    $database->members = $members;
}

[Pixa1]

Thank you very much @stevebauman, you pointed me in the right direction. I ended up with this code

public function handle(LdapGroup $ldap, DatabaseGroup $database)
    {
        $members = [];

        $ldap->members()->each(function ($member) use (&$members) {
            if ($member instanceof LdapUser || $member instanceof LdapGroup) {
                $members[] = $member->getDn();
            }
        });
        $database->members = $members;
        $database->name = $ldap->getFirstAttribute('cn');
        $database->email = $ldap->getFirstAttribute('mail');
        $database->members = $members;
        $database->managedby = $ldap->getFirstAttribute('managedby');
        $database->msexchcomanagedbylink = $ldap->getAttribute('msexchcomanagedbylink');
        $database->managedobjects = $ldap->getAttribute('managedobjects');
        $database->distinguishedname = $ldap->getFirstAttribute('distinguishedname');
    } 

I really appreciate your help and your contribution.

[Pixa1]

Hi, I just wanted to come by and say that the method from the previous post took extremely long to execute, (a lot of AD groups with large set of members) so I created another function to handle LDAP queries:

class GroupAttributeHandler extends Model
{

    public function handle(LdapGroup $ldap, DatabaseGroup $database)
    {
        $members = [];
        if (count(preg_grep('/^member;range=[\d]*/', array_keys($ldap->getAttributes()))) > 0) {
            $members = $this->getAllMembers($ldap->getDn());
            $database->members = array_values($members);
        } else {
            $database->members = $ldap->member;
        }
        $database->name = $ldap->getFirstAttribute('cn');
        $database->email = $ldap->getFirstAttribute('mail');
        $database->managedby = $ldap->getFirstAttribute('managedby');
        $database->msexchcomanagedbylink = $ldap->getAttribute('msexchcomanagedbylink');
        $database->managedobjects = $ldap->getAttribute('managedobjects');
        $database->distinguishedname = $ldap->getFirstAttribute('distinguishedname');
    }

    protected function getAllMembers($group)
    {
        $rangeStart = 0;
        $rangeSize = 1500;
        $members = [];

        $ldap = Container::getConnection();
        do {
            $range = "$rangeStart-" . ($rangeStart + $rangeSize - 1);
            try {
                $result = $ldap->query()->where('distinguishedname', '=', $group)->select(["member;range=$range"])->first();
            } catch (Exception $e) {
                error_log("LDAP query failed: " . $e->getMessage());
                break;
            }

            $newRange = preg_grep('/^member;range=[\d]*/', array_keys($result));
            $currentMembers = $result[$newRange[2]];
            if ($currentMembers) {
                $members = array_merge($members, $currentMembers);
                $rangeStart += $rangeSize;
            }else {
                break; // No more members to fetch
            }
        } while ("member;range=$range" == $newRange[2]);

        return ($members);
    }
}

This method synchronizes more than 8000 groups, some of them have 3000+ members (around 100), in 40 seconds, while the other approach took more than 10 minutes.

p.s. I'm not a pro developer, so the code above could use some refactoring and/or better error handling.