search

Adambean / gitlab-ce-ldap-sync

Synchronise users and groups (including group members) from an LDAP instance with Gitlab CE (and EE in free tier) self-hosted instance(s).
Apache License 2.0
59 stars 23 forks source link

Crash when encountering subgroups #6

Closed jsiegers closed 5 years ago

jsiegers commented 5 years ago

I've got a weird crash. I've got some groups that a user makes. One of them is called 'A Folder Renamed'.

This is what I get in the log:

LDAP users and groups sync script for Gitlab-CE

[notice] Loading configuration.
[notice] Loaded configuration.
[notice] Validating configuration.
[notice] Validated configuration.
[notice] Retrieving directory users and groups.
[notice] Establishing LDAP connection.
[notice] LDAP connection established.
[notice] 6 directory user(s) found.
[info] User "gitlab-ldap-bind-user" in ignore list.
[info] Found directory user "postlab-cloud" [uid=postlab-cloud,cn=users,cn=accounts,dc=fipa01,dc=postlab,dc=cloud].
[info] User "gl-ams-01" in ignore list.
[info] Found directory user "jsiegers" [uid=jsiegers,cn=users,cn=accounts,dc=fipa01,dc=postlab,dc=cloud].
[info] Found directory user "testuser" [uid=testuser,cn=users,cn=accounts,dc=fipa01,dc=postlab,dc=cloud].
[notice] 3 directory user(s) recognised.
[notice] 6 directory group(s) found.
[info] Found directory group "test-team".
[info] Group #1 / member #1: User "gl-ams-01" in ignore list.
[info] Found directory group "test-team" member "jsiegers".
[info] Found directory group "test-team" member "postlab-cloud".
[info] Found directory group "test-team" member "testuser".
[notice] 3 directory group "test-team" member(s) recognised.
[info] Found directory group "test-group".
[info] Found directory group "test-group" member "jsiegers".
[info] Group #2 / member #2: User "gl-ams-01" in ignore list.
[info] Found directory group "test-group" member "postlab-cloud".
[info] Found directory group "test-group" member "testuser".
[notice] 3 directory group "test-group" member(s) recognised.
[info] Group "postlab-cloud" in ignore list.
[info] Found directory group "gitlab-admins".
[info] Group "gitlab-admins" members are administrators.
[info] Found directory group "gitlab-admins" member "postlab-cloud".
[info] Group #4 / member #1: User "postlab-cloud" is an administrator.
[info] Group #4 / member #2: User "gl-ams-01" in ignore list.
[notice] 1 directory group "gitlab-admins" member(s) recognised.
[info] Found directory group "gitlab-externals".
[info] Group "gitlab-externals" members are external.
[info] Group #5 / member #1: User "gl-ams-01" in ignore list.
[notice] 0 directory group "gitlab-externals" member(s) recognised.
[notice] 4 directory group(s) recognised.
[notice] LDAP connection closed.
[notice] Retrieved directory users and groups.
[notice] Deploying users and groups to Gitlab instances.
[notice] Establishing Gitlab connection.
[notice] Finding all existing Gitlab users...
[info] Found Gitlab user #4 "testuser".
[info] Found Gitlab user #3 "jsiegers".
[info] Found Gitlab user #2 "postlab-cloud".
[info] Gitlab built-in root user will be ignored.
[notice] 3 Gitlab user(s) found.
[notice] Creating directory users of which don't exist in Gitlab...
[notice] 0 Gitlab user(s) created.
[notice] Disabling Gitlab users of which don't exist in directory...
[notice] 0 Gitlab user(s) disabled.
[notice] Updating users of which were already in both Gitlab and the directory...
[info] Updating Gitlab user #3 "jsiegers".
[info] Updating Gitlab user #2 "postlab-cloud".
[info] Updating Gitlab user #4 "testuser".
[notice] 3 Gitlab user(s) updated.
[notice] Finding all existing Gitlab groups...
[info] Found Gitlab group #8 "A Folder Renamed" [a-folder-renamed].
[info] Found Gitlab group #9 "A Sub Folder" [a-sub-folder].
[info] Found Gitlab group #3 "gitlab admins" [gitlab-admins].
[info] Found Gitlab group #11 "Series" [series].
[info] Found Gitlab group #5 "test group" [test-group].
[info] Found Gitlab group #7 "test team" [test-team].
[info] Found Gitlab group #10 "The Second Sub Folder" [the-second-sub-folder].
[notice] 7 Gitlab group(s) found.
[notice] Creating directory groups of which don't exist in Gitlab...
[warning] Not creating Gitlab group "gitlab externals" [gitlab-externals]: No members in directory group, or config gitlab->options->createEmptyGroups is disabled.
[notice] 0 Gitlab group(s) created.
[notice] Deleting Gitlab groups of which don't exist in directory...
[info] Not deleting Gitlab group #8 "A Folder Renamed" [a-folder-renamed]: Has members in directory group, or config gitlab->options->deleteExtraGroups is disabled.
[info] Not deleting Gitlab group #9 "A Sub Folder" [a-sub-folder]: Has members in directory group, or config gitlab->options->deleteExtraGroups is disabled.
[info] Not deleting Gitlab group #11 "Series" [series]: Has members in directory group, or config gitlab->options->deleteExtraGroups is disabled.
[info] Not deleting Gitlab group #10 "The Second Sub Folder" [the-second-sub-folder]: Has members in directory group, or config gitlab->options->deleteExtraGroups is disabled.
[notice] 0 Gitlab group(s) deleted.
[notice] Updating groups of which were already in both Gitlab and the directory...
[info] Updating Gitlab group #8 "A Folder Renamed" [a-folder-renamed].
[info] Gitlab group "A Folder Renamed" has no LDAP details available.
[info] Updating Gitlab group #9 "A Sub Folder" [a-sub-folder].
[info] Gitlab group "A Sub Folder" has no LDAP details available.
[info] Updating Gitlab group #11 "Series" [series].
[info] Gitlab group "Series" has no LDAP details available.
[info] Updating Gitlab group #10 "The Second Sub Folder" [the-second-sub-folder].
[info] Gitlab group "The Second Sub Folder" has no LDAP details available.
[info] Updating Gitlab group #3 "gitlab admins" [gitlab-admins].
[info] Updating Gitlab group #5 "test group" [test-group].
[info] Updating Gitlab group #7 "test team" [test-team].
[notice] 3 Gitlab group(s) updated.
[notice] Synchronising Gitlab group members with directory group members...
PHP Notice:  Undefined index: A Folder Renamed in /usr/local/scripts/gitlab-ce-ldap-sync/src/LdapSyncCommand.php on line 1362
PHP Fatal error:  Uncaught TypeError: Argument 2 passed to AdamReece\GitlabCeLdapSync\LdapSyncCommand::in_array_i() must be of the type array, null given, called in /usr/local/scripts/gitlab-ce-ldap-sync/src/LdapSyncCommand.php on line 1362 and defined in /usr/local/scripts/gitlab-ce-ldap-sync/src/LdapSyncCommand.php:1513
Stack trace:
#0 /usr/local/scripts/gitlab-ce-ldap-sync/src/LdapSyncCommand.php(1362): AdamReece\GitlabCeLdapSync\LdapSyncCommand->in_array_i('jsiegers', NULL)
#1 /usr/local/scripts/gitlab-ce-ldap-sync/src/LdapSyncCommand.php(144): AdamReece\GitlabCeLdapSync\LdapSyncCommand->deployGitlabUsersAndGroups(Array, 'gitlab', Array, Array, 3, Array, 4)
#2 /usr/local/scripts/gitlab-ce-ldap-sync/vendor/symfony/console/Command/Command.php(255): AdamReece\GitlabCeLdapSync\LdapSyncCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#3 /usr/local/scripts/gitlab-ce-ldap-sync/vendor/symfony/console/Application.php(946): Symfony\Component\Console\Command\C in /usr/local/scripts/gitlab-ce-ldap-sync/src/LdapSyncCommand.php on line 1513
Adambean commented 5 years ago

if (!$this->in_array_i($gitlabUserName, $ldapGroupsSafe[$gitlabGroupName])) {

Looks like it's trying to process a group that doesn't exist. To get more insight could you please insert this block after the foreach condition at line 1361?

                if (!isset($ldapGroupsSafe[$gitlabGroupName]) || !is_array($ldapGroupsSafe[$gitlabGroupName])) {
                    $this->logger->error(sprintf("Group \"%s\" doesn't appear to exist at path \"%s\".", $gitlabGroupName, $gitlabGroupPath));
                    continue;
                }

You'll then see an error output as to which group is troubling you.

jsiegers commented 5 years ago

It now git this (just an excerpt of the log): 

[error] Group "A Folder Renamed" doesn't appear to exist at path "a-folder-renamed".
[error] Group "A Folder Renamed" doesn't appear to exist at path "a-folder-renamed".
[error] Group "A Folder Renamed" doesn't appear to exist at path "a-folder-renamed".
[notice] Synchronising 0 member(s) for group #8 "A Folder Renamed" [a-folder-renamed]...
[notice] Finding existing group members...
[notice] 0 Gitlab group "A Folder Renamed" [a-folder-renamed] member(s) found.
[notice] Adding missing group members...
[notice] 0 Gitlab group "A Folder Renamed" [a-folder-renamed] member(s) added.
[notice] Deleting extra group members...
[notice] 0 Gitlab group "A Folder Renamed" [a-folder-renamed] member(s) deleted.
[error] Group "A Sub Folder" doesn't appear to exist at path "a-sub-folder".
[error] Group "A Sub Folder" doesn't appear to exist at path "a-sub-folder".
[error] Group "A Sub Folder" doesn't appear to exist at path "a-sub-folder".
[notice] Synchronising 0 member(s) for group #9 "A Sub Folder" [a-sub-folder]...
[notice] Finding existing group members...
[notice] 0 Gitlab group "A Sub Folder" [a-sub-folder] member(s) found.
[notice] Adding missing group members...
[notice] 0 Gitlab group "A Sub Folder" [a-sub-folder] member(s) added.
[notice] Deleting extra group members...
[notice] 0 Gitlab group "A Sub Folder" [a-sub-folder] member(s) deleted.
[error] Group "Series" doesn't appear to exist at path "series".
[error] Group "Series" doesn't appear to exist at path "series".
[error] Group "Series" doesn't appear to exist at path "series".
[notice] Synchronising 0 member(s) for group #11 "Series" [series]...
[notice] Finding existing group members...
[notice] 0 Gitlab group "Series" [series] member(s) found.
[notice] Adding missing group members...
[notice] 0 Gitlab group "Series" [series] member(s) added.
[notice] Deleting extra group members...
[notice] 0 Gitlab group "Series" [series] member(s) deleted.
[error] Group "The Second Sub Folder" doesn't appear to exist at path "the-second-sub-folder".
[error] Group "The Second Sub Folder" doesn't appear to exist at path "the-second-sub-folder".
[error] Group "The Second Sub Folder" doesn't appear to exist at path "the-second-sub-folder".

These groups are subgroups that have been created within the main group. What I do with the GitLab API is look only for groups where the parent_id is missing. Those are the groups at root level.

At least as I understand this script is that it only creates groups at root level and should ignore the subgroups.

jsiegers commented 5 years ago

btw with these extra lines it doesn't crash anymore.

Adambean commented 5 years ago

subgroups

That would be why. I've not built any special sub-group handling (or planned to really), but I would have anticipated that this tool would still sync groups though lose their parental hierarchy.

Are these really sub-groups in your directory, or have you made these separately (or moved synced ones into children) on the Gitlab side?

jsiegers commented 5 years ago

Yes these are really subgroups. In our situation users are allowed to create their own subgroup structure; so there will be a lot of subgroups.

jsiegers commented 5 years ago

Any thoughts on this on yet? (I like to keep it hot ;-)

Adambean commented 5 years ago

Sub-groups wasn't in my planning as none of my directories are organised like this. I don't really have time to implement that at present to implement this so someone else will likely have to take it on.

jsiegers commented 5 years ago

I understand. But all I want is that the script doesn't crash when it encounters a subgroup. That piece of code you mentioned earlier i think did the job but is not in the official release yet.

Adambean commented 5 years ago

Yep that's fair. That's been sent in. ^

jsiegers commented 5 years ago

Cheers! Doesn't crash now!