Move away from LDAP - Githubissues

mlehotskylf commented 3 months ago

Move away from LDAP

Docs:

https://docs.google.com/document/d/16Okl4PAewO67CFcqknO_jGnpAEcLoDfv0pTXQhLPcVk/edit#heading=h.hallpd1bjve8
https://docs.google.com/document/u/0/d/1DH4mAfQTDh4xRIZlJrDpyQHa5T7vKfJkAlvIBJDWPig/edit
[x] 1) Investigate SSO (SAML) support for ITX Gerrit test instance (which lives in ITX DEV) - create client, etc Jordan Evans - ticket is here
[x] 2) - in parallel with 1: Create “is LFID authorized for CLA Group ID?” endpoint in EasyCLA Harold Wanyama
[x] 3) Remove outbound “group sync” for Gerrit contributor flow (independent of 2) Harold Wanyama
[x] 4) dependent on 2: create Auth0 lookup of authorized status Eric Searcy
[ ] 5) test in dev gerrit: dependent on 1, 2, 3, and 4 Veerendra Singh Thakur + others supporting (Juansebastion?)
[ ] 6) remove ICLA / CCLA onboarding - coordination effort between Harold Wanyama and PCC team (Jordan) – not sure if this is required before testing or not – could just put in bogus values as needed for dev onboarding for test purposes if 1—4 are done first
[ ] 7) prep for prod deployment: get inventory of existing Gerrit-connected CLA groups and port their configuration to SSO lookups Eric Searcy with support from others (Juansebastian Arias?)

emsearcy commented 3 months ago

There is no "migration" of users from LDAP to Auth0 as part of this, and the second link is not very relevant to EasyCLA, it's just another part of the overarching project.

What's principally needed from EasyCLA is: 1) stop taking/storing "ICLA/CCLA node IDs" in the onboarding APIs and PCC UI, and remove any reference to setting these up in the EasyCLA and/or PCC docs 2) change the onboarding prompts in PCC to show the EasyCLA "CLA group" IDs which need configuration by an Auth0 administrator configuration (in addition to the Gerrit snippet configurations), 3) stop attempting to push ICLA and ECLA participants to the Drupal Identity API, and 4) share with me the API calls needed to fetch ICLA and/or ECLA signature status by LDAP username lookup (multiple calls are OK, but should take less than 5 seconds total for all calls--so pagination will likely not be supported if the time is theoretically unbounded, and we'd need a specific username-lookup endpoint in that case).

nickmango commented 3 months ago

Much appreciated @emsearcy for the clarifications and this is clearer, I had a couple of questions on 2 & 4. 2) change the onboarding prompts in PCC to show the EasyCLA "CLA group" IDs which need configuration by an Auth0 administrator configuration (in addition to the Gerrit snippet configurations). I suppose the configurations would be handled by Easycla API endpoint on onboarding ? If yes , could you detail these configurations needed to be processed and if theres any external calls (same idea we use for Github) 4) share with me the API calls needed to fetch ICLA and/or ECLA signature status by LDAP username lookup (multiple calls are OK, but should take less than 5 seconds total for all calls--so pagination will likely not be supported if the time is theoretically unbounded, and we'd need a specific username-lookup endpoint in that case). Would this be to support backward compatability with our existing LDAP users. And moving forward this will remain the same approach much as we are shifting from LDAP ?

Thanks, can we have a session to discuss the overall approach as well such that PCC and the Easycla integration is well aligned ? cc @mlehotskylf

nickmango commented 3 months ago

@emsearcy the implemented the lfid authorized endpoint https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#tag/signatures/operation/isAuthorized

nickmango commented 3 months ago

@emsearcy I have decommissioned the group sync feature for the gerrit flow by removing the existing logic that added icla,ecla users to the LDAP group. However theres a use case when we add/remove approval list by email . In this case if the user was a gerrit user we added/removed the user to the LDAP group upon updating the approval list items (This happens in the lfx company dashboard for easycla). Based on this update, are we able to integrate auth0 update of adding or removing a gerrit user ? cc @mlehotskylf

emsearcy commented 3 months ago

As long as those users will subsequently return "true" in the "isAuthorized" endpoint, it should function identically from the end users perspective to the group-add flow. (groups were only synced into Gerrit from LDAP on their next login, anyhow, same as when /cla/authorization will be checked)

nickmango commented 3 months ago

well noted !

amolsontakke3576 commented 2 months ago

@nickmango is it deployed to dev ? I got below error .../cla-group/01af041c-fa69-4052-a23c-fb8c1d3bef24/project/lfhR1vwaB7uuX5ExzO/gerrit Request Method: POST Payload : {"gerritName":"ONAP","gerritUrl":"https://gerrit.onap.org/","version":"v1"} Response :

{
    "status": 400,
    "stack": "",
    "details": null,
    "message": "should specify at least a LDAP group for ICLA or CCLA",
    "code": "error",
    "requestId": "7e67535a-3424-4667-9eef-a4400f4b6c7a",
    "data": {
        "gerritName": "ONAP",
        "gerritUrl": "https://gerrit.onap.org/",
        "version": "v1"
    }
}

emsearcy commented 2 months ago

just for reference, here is the script I wrote for doing comparisons between group membership and EasyCLA API status: https://github.com/LF-Engineering/easycla-gerrit-compare-script/blob/main/compare_authorizations.py

nickmango commented 2 months ago

@nickmango is it deployed to dev ? I got below error .../cla-group/01af041c-fa69-4052-a23c-fb8c1d3bef24/project/lfhR1vwaB7uuX5ExzO/gerritRequest Method: POST Payload : {"gerritName":"ONAP","gerritUrl":"https://gerrit.onap.org/","version":"v1"} Response :
{
    "status": 400,
    "stack": "",
    "details": null,
    "message": "should specify at least a LDAP group for ICLA or CCLA",
    "code": "error",
    "requestId": "7e67535a-3424-4667-9eef-a4400f4b6c7a",
    "data": {
        "gerritName": "ONAP",
        "gerritUrl": "https://gerrit.onap.org/",
        "version": "v1"
    }
}
@amolsontakke3576 this should be good. I had reverted this when we were to move to prod but have restored this functionality. Kindly review

emsearcy commented 2 months ago

@mlehotskylf as discussed in Slack -- QA fail for #2 while working on #7 (which is otherwise ready). Please see following discrepancies between EasyCLA-managed LDAP groups, and the live API responses. I need help from the EasyCLA team to identify the cause of the discrepancies and determine a path forward, possibly with support from the RelEng team.

My expectations:

Previous invalidations (ICLA invalidations, CCLA invalidations, or removal of employees from CCLA lists) of Gerrit-linked groups should have been synced as LDAP group removals: if that wasn't done in the past, that's not ideal, but the good news is that moving to the API will solve it for us! (Even so, it might not be the only source of the discrepancies...needs confirmation!)
Out-of-band signatures (including migration of pre-EasyCLA authorizations) should have been added to the EasyCLA DB via the established process.
Support & engineering staff should be "unauthorized", but RelEng team has a workaround that adds an override on the Gerrit side. Anyone previously getting this type of override via LDAP group memberrship needs to be migrated to the Gerrit group-membership-override (nested group setup).
Product support & RelEng teams should review the solution(s) to ensure they understand the amount of potential user disruption / support burden due to any outstanding user transitions from authorized to unauthorized.

ONAP: 229 users authorized, 653 unauthorized, 15 errors OPNFV: 68 users authorized, 42 unauthorized, 0 errors ORAN-SC (OSS): 153 users authorized, 154 unauthorized, 1 errors ORAN-ASC: 30 users authorized, 15 unauthorized, 0 errors

amolsontakke3576 commented 2 months ago

@nickmango is it deployed to dev ? I got below error .../cla-group/01af041c-fa69-4052-a23c-fb8c1d3bef24/project/lfhR1vwaB7uuX5ExzO/gerritRequest Method: POST Payload : {"gerritName":"ONAP","gerritUrl":"https://gerrit.onap.org/","version":"v1"} Response :
{
    "status": 400,
    "stack": "",
    "details": null,
    "message": "should specify at least a LDAP group for ICLA or CCLA",
    "code": "error",
    "requestId": "7e67535a-3424-4667-9eef-a4400f4b6c7a",
    "data": {
        "gerritName": "ONAP",
        "gerritUrl": "https://gerrit.onap.org/",
        "version": "v1"
    }
}

@nickmango It again reproduced on dev.

amolsontakke3576 commented 2 months ago

@nickmango any updates? I can still see the same errors.

nickmango commented 2 months ago

@amolsontakke3576 with the latest updates I reverted this section. There was a window it was ready for testing.

nickmango commented 2 months ago

@amolsontakke3576 kindly retest

communitybridge / easycla

Move away from LDAP #4348