[2024.06 and earlier] Group members not synced to RES during AD sync

[HimajaChandaluri]

Bug description

Group members will not properly sync to RES if the GroupOU differs from the UserOU.

RES creates an ldapsearch filter when attempting to sync users from an AD group. The current filter incorrectly utilizes the UserOU parameter instead of the GroupOU parameter. The result is that the search fails to return any users. This behavior only occurs in instances where the UsersOU and GroupOU differ.

Affected versions

This issue affects all RES versions 2024.06 or earlier

Mitigation

Please follow the following steps to resolve the issue,

1. Follow the steps below to download the patch.py script and group_member_sync_bug_fix.patch file by replacing <output-directory> and <res_version> in the command below with the local directory you’d like to download the patch script and patch file to:
   • The patch script requires AWS CLI v2, Python 3.9.16 or above, and Boto3.
     • Configure the AWS CLI for the account and region where RES is deployed, and make sure that you have S3 permissions to write to the bucket created by RES.
     • The patch only supports RES versions 2024.04.02 and 2024.06. If you are using 2024.04 or 2024.04.01, you can follow the steps listed in the public document to first update your environment to 2024.04.02 prior to applying the patch.

RES Version	Patch download link
RES 2024.04.02	2024.04.02_group_member_sync_bug_fix.patch
RES 2024.06	2024.06_group_member_sync_bug_fix.patch

OUTPUT_DIRECTORY=<output-directory>
RES_VERSION=<res_version>
mkdir -p ${OUTPUT_DIRECTORY}

curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patch.py --output ${OUTPUT_DIRECTORY}/patch.py

curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patches/${RES_VERSION}_group_member_sync_bug_fix.patch --output ${OUTPUT_DIRECTORY}/${RES_VERSION}_group_member_sync_bug_fix.patch

2. Navigate to the directory where the patch script and patch file are downloaded. Run the following patch command by replacing :

cd ${OUTPUT_DIRECTORY}
ENVIRONMENT_NAME=<environment-name>

python3 patch.py --environment-name ${ENVIRONMENT_NAME} --res-version ${RES_VERSION} --module cluster-manager --patch $PWD/${RES_VERSION}_group_member_sync_bug_fix.patch

3. Restart the cluster-manager instance for your environment using the commands below:

INSTANCE_ID=$(aws ec2 describe-instances \
    --filters \
    Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
    Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
    --query "Reservations[0].Instances[0].InstanceId" \
    --output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}

[gibso221]

Hi @HimajaChandaluri,

This is still an issue in RES 2024.10 - the live code on the Cluster Manager instance shows it filters users on ldap_user_base rather than ldap_group_base, which the patch file fixes. This also means I have to manually adjust the patching script to allow for the newer RES version, since it only accounts for RES 2024.06/2024.04.02

There's now a new function name called fetch_ldap_users_in_group (formally fetch_all_ldap_users_in_group) which means I'll probably have to modify the patch file too.

Are there plans to fix this soon?

Thanks

[root@ip-1-2-3-4 adsync]# pwd
/opt/idea/python/latest/lib/python3.9/site-packages/ideaclustermanager/app/adsync
[root@ip-1-2-3-4 adsync]# egrep -A 10 "ldap_users_in_group" adsync_service.py
    def fetch_ldap_users_in_group(self, ldap_group_name: str, users_filter: Optional[str]) -> list[dict]:
        ldap_base = self.context.accounts.ldap_client.ldap_base
        filterstr = (
            f"(memberOf=cn={ldap_group_name},{self.context.accounts.ldap_client.ldap_user_base})"
        )
...

The patch file for 2024.06:

--- idea-cluster-manager-2024.6/ideaclustermanager/app/adsync/adsync_service.py 2024-06-11 16:15:53
+++ idea-cluster-manager-2024.6/ideaclustermanager/app/adsync/adsync_service.py 2024-06-11 16:32:45
@@ -108,7 +108,7 @@
     def fetch_all_ldap_users_in_group(self, ldap_group_name) -> list[dict]:
         ldap_base = self.context.accounts.ldap_client.ldap_base
         filter_str = (
-            f"(&(objectClass=user)(memberOf=cn={ldap_group_name},{self.context.accounts.ldap_client.ldap_user_base}))"
+            f"(&(objectClass=user)(memberOf=cn={ldap_group_name},{self.context.accounts.ldap_client.ldap_group_base}))"
         )
         ldap_users, _ = self.context.accounts.ldap_client.search_users(
             username_filter=SocaFilter(), ldap_base=ldap_base, filter_str=filter_str

[nguyen102]

Hi,

What version of RES would you like to deploy? We can provide a patch for that version.

Thanks, Tim

[gibso221]

Hi Tim,

Could you please release a patch for the latest version, 2024.10?

Thanks, Andrew

[nguyen102]

Hi Andrew,

I took a look at the CR for .10 release and the method name fetch_all_ldap_users_in_group has been renamed to fetch_ldap_users_in_group. Based on that I believe this patch should fix the .10 release. Let me know if you have any further questions.

2024_10_group_member_sync_bug_fix_patch.patch

Thanks, Tim