search

janus-idp / backstage-plugins

Plugins for Backstage
https://janus-idp.io
Apache License 2.0
151 stars 150 forks source link

Keycloak backend plugin is not working as expected #2112

Closed paralika closed 3 weeks ago

paralika commented 3 months ago

Describe the bug

I have setup the Keycloak backend plugin by following the steps given on below this link - https://janus-idp.io/plugins/keycloak/ . Plugin configure successfully and it started syncing users and groups, but then abruptly, it stopped syncing with below error.

Processor BuiltinKindsEntityProcessor threw an error while validating the entity group:default/mainstream-engineering-admin; caused by TypeError: /spec/members/2 must be string - type: string entity=group:default/mainstream-engineering-admin location=url:https://authusqa.devops.sws.company.com/auth/admin/realms/chim/groups/480049ad-7a33-41ab-87a6-382a797831e0

I am using the new backend and followed the plugin configuration steps accordingly.

Expected Behavior

All users and groups from keyclock realm should be synced

What are the steps to reproduce this bug?

  1. Setup backstage
  2. Configure the Keycloak plugin for new backup configuration (using steps from https://janus-idp.io/plugins/keycloak/)

Versions of software used and environment

@janus-idp/backstage-plugin-keycloak-backend": "^1.13.1",

Backstage - 1.30.4 (create-app@0.5.18)

paralika commented 2 months ago

Any comments here please?

kadel commented 2 months ago

@paralika This looks like an invalid group definition. Is the mainstream-engineering-admin group something that you created, or is it imported by the plugin?

Can you share what this group looks like? Find this group in Catalog, lick on 3 dots in top right corner, and select inspect entity, there you can find Raw Yaml.

paralika commented 2 months ago

@kadel "mainstream-engineering-admin" is available in Keycloak. The plugin is trying to import it, but it is not successful yet. I have ~1k groups in Keycloak. Out of that, only 341 groups were imported successfully and for others it is failing with this error.

kadel commented 2 months ago

The error message complains about a third member of that group. Can you check the members of that group? Maybe you notice something suspicious. For example, isn't there a user with numeric username?

paralika commented 2 months ago

@kadel I do not see any suspicious chars. Please see the screenshot below. The username is an alphanumeric value pointing to the users.

image

04kash commented 2 months ago

The Keycloak admin API returns usernames as strings (https://github.com/keycloak/keycloak/blob/main/js/libs/keycloak-admin-client/src/defs/userRepresentation.ts#L10), so I don't think that should be an issue.

I tested by assigning both numeric and alphanumeric usernames to a group, and the sync process worked as expected for users and groups. Here is what I received under spec.members:

spec:
  members:
    - "078687777"
    - z007abc7666666

Screenshot from 2024-09-04 16-38-24

Are you using any custom transformers in your setup? If so, it's possible that they might be mutating the entity in a way that’s causing type issues during validation.

paralika commented 2 months ago

No, I do not have any custom transformer set. I am following the instructions given at - https://janus-idp.io/plugins/keycloak/

Did you follow the same instructions? Wondering if I missed anything.

JohannesWill commented 1 month ago

I figured out 2 problems:

  1. The groups.members hat entries wich are not in parsedUsers https://github.com/janus-idp/backstage-plugins/blob/58d2baa1073428a77e5c723c8bad1c24e83fd440/plugins/keycloak-backend/src/lib/read.ts#L300

  2. Members are missing when a group has many members (> options?.userQuerySize) https://github.com/janus-idp/backstage-plugins/blob/58d2baa1073428a77e5c723c8bad1c24e83fd440/plugins/keycloak-backend/src/lib/read.ts#L229

AndrienkoAleksandr commented 1 month ago
  1. The groups.members hat entries wich are not in parsedUsers https://github.com/janus-idp/backstage-plugins/blob/58d2baa1073428a77e5c723c8bad1c24e83fd440/plugins/keycloak-backend/src/lib/read.ts#L300

@JohannesWill Good catch, I agree with that. The current code could add an undefined value to the group entity member list. Then the plugin will attempt to commit this group to the catalog backend and fail with a TypeError during member list validation. I believe that with a very large database, the Keycloak plugin fetches users using pagination. However, one or more user fetch requests could fail. The current code skips this error and doesn't log it. As a result, the parsed user list will be incomplete, and the group member list could contain undefined values for some users.

  1. Members are missing when a group has many members (> options?.userQuerySize) https://github.com/janus-idp/backstage-plugins/blob/58d2baa1073428a77e5c723c8bad1c24e83fd440/plugins/keycloak-backend/src/lib/read.ts#L229

Good catch too, but I think we need to fix it like separated issue.

AndrienkoAleksandr commented 1 month ago

Members are missing when a group has many members (> options?.userQuerySize)

@JohannesWill, thanks. I created issue: https://issues.redhat.com/browse/RHIDP-4391 . Fix is in the road: https://github.com/janus-idp/backstage-plugins/pull/2324

JohannesWill commented 1 month ago

@JohannesWill Good catch, I agree with that. The current code could add an undefined value to the group entity member list. Then the plugin will attempt to commit this group to the catalog backend and fail with a TypeError during member list validation. I believe that with a very large database, the Keycloak plugin fetches users using pagination. However, one or more user fetch requests could fail. The current code skips this error and doesn't log it. As a result, the parsed user list will be incomplete, and the group member list could contain undefined values for some users.

In our case, we have 8 users (service-account-*), which are not included in 'parsedUsers'

AndrienkoAleksandr commented 1 month ago

In our case, we have 8 users (service-account-*), which are not included in 'parsedUsers'

Oh, make sense, I even didn't know about this Keycloak feature...

JohannesWill commented 4 weeks ago

Fixed with https://github.com/janus-idp/backstage-plugins/pull/2319 Tested it with "@janus-idp/backstage-plugin-keycloak-backend": "2.0.8"

paralika commented 3 weeks ago

Thanks @JohannesWill and @AndrienkoAleksandr. It's working.

nickboldt commented 3 weeks ago

Apparently fixed in 1.3.1 with update to "@janus-idp/backstage-plugin-keycloak-backend": "1.13.4" in https://github.com/janus-idp/backstage-showcase/pull/1823/files

Related update in 1.3 branch of plugins is https://github.com/janus-idp/backstage-plugins/pull/2429/files#diff-9a8714e6876ad3175b960df6dbacd41b4c818c4dee1617fef0d9e5202e847928R2 ... which is a MAJOR bump, not a patch one?

Ah but then fixed with https://github.com/janus-idp/backstage-plugins/commit/b3835da8bc84edc762e36b81857296fca085b109 to move to 1.13.4

So... I think we're good here! for both 1.3.z and 1.4.