search

communitybridge / easycla

The Contributor License Agreement (CLA) service of the Linux Foundation lets project contributors read, sign, and submit contributor license agreements easily.
https://easycla.lfx.linuxfoundation.org
MIT License
64 stars 45 forks source link

migrate repositories api to golang #798

Closed prasannamahajan closed 2 years ago

prasannamahajan commented 4 years ago

Summary

Several of the APIs used by the Project management console need migration to golang to support 1) consolidated queries and 2) LFX V2 API integration.

Background

Migrate the following Python APIs to Golang (v3): POST /v1/repository DELETE /v1/repository/ GET /v1/project//configuration_orgs_and_repos

Provide Golang implementation for two endpoints:

User Story

As a developer, I want to optimize the PMC queries. As a developer, I want to consolidate API requests to the golang backend API. As a developer, I want to support LFX v2 integration by using golang to invoke the legacy v3 API

Acceptance Criteria

The "done" criteria when this feature or problem is resolved. Such as:

  1. Unit Tests added and running in CI
  2. Functional Tests updated to cover feature, if applicable
  3. Demonstrate the set of capabilities to the product team while the code is running in the STAGING environment.

References

(Optional) Provide any code or specification references that would be helpful for the developer implementing this feature.

prasannamahajan commented 4 years ago

Pull request: https://github.com/communitybridge/easycla/pull/815

nirupamav commented 4 years ago

Tested and observed below issues POST ACTION

  1. Observed 401 is returned when invalid external projectid is provided. 404/400 should be returned to be consistent with other V3 api's image
  2. Able to add invalid github details image
  3. Able to add when invalid projectId(CLA Group Id) is provided image
  4. Able to add without repository name image

image

  1. 500 is returned instead of 401 when authorization token is expired image

GET ACTION

  1. 500 is returned instead of 401 when authorization token is expired image
  2. Observed 401 is returned when invalid external projectid is provided. 404/400 should be returned to be consistent with other V3 api's image

DELETE ACTION

  1. 200 is returned instead of 204 image
  2. Observed 401 is returned when invalid external projectid is provided. 404/400 should be returned to be consistent with other V3 api's image
  3. 500 is returned instead of 401 when authorization token is expired image

IN UI still V1 api's are used for GET, POST and DELETE image

nirupamav commented 3 years ago

Tested and observed that below issues are still reproducible.

POST ACTION 5. 500 is returned instead of 401 when authorization token is expired image.png

6. Not able to add Repositories even when valid details are provided. Due to this issue testing of issues 2,3,4 are blocked image.png

image.png

GET ACTION 1. 500 is returned instead of 401 when authorization token is expired image.png

DELETE ACTION 3. 500 is returned instead of 401 when authorization token is expired image.png

IN UI still V1 api's are used for GET, POST and DELETE image.png

rinkeshbhutwala commented 3 years ago

@nirupamav For Issue no. 6 I have verified that Data is not stored properly in cla tables or it might be wrong data in tables that's why error is coming cla-group is not linked with given project SFID Can you please check with fresh data i.e. create a new cla-group and try the whole workflow?

nirupamav commented 3 years ago

@rinkeshbhutwala It is the correct data. I am using it in V1 for testing

dealako commented 3 years ago

@makkalot can you review the 500 token expired error? Is this related to the GH session token? Can we determine if the token is expired and possibly refresh?

Relates to #781

dealako commented 3 years ago

Summary

@amolsontakke3576 re-assigning this task to you. I did a quick search in the v1 project console UI. If you review the cla.service.ts /v1 API calls and perform a "find usages" search, you will notice that we still have a number of functions that call the v1 backend API. We need to replace these endpoint API calls with /v3 endpoint API calls.

Screen Shot 2020-10-05 at 1.50.50 PM.png

Tasks

Related to #781

amolsontakke3576 commented 3 years ago

@David @rinkeshbhutwala I have verified Project console endpoints and below are a few that still use V1.

  1. getAllProjectsFromSFDC : /v1/salesforce/projects
  2. getProjectFromSFDC : /v1/salesforce/project?id=
  3. getProjectConfigurationAndRepos : /v1/project/' + projectId + '/configuration_orgs_and_repos

Below are few endpoints in the Corporate console still uses V1

  1. putSignature : /v1/signature
  2. postCompany : /v1/company
  3. getCLAManagers : /v1/signature/' + signatureId + '/manager
  4. getCompanyUnsignedProjects : /v1/company/' + companyId + '/project/unsigned
  5. postCorporateSignatureRequest : /v1/request-corporate-signature

Below are few endpoints in the Contributor console still uses V1

  1. postOrGetUserForGerrit : /v1/user/gerrit
  2. getGerritProjects : /v1/project/' + projectId + '/gerrits

Can you please suggest equivalent endpoints in Go lang?

rinkeshbhutwala commented 3 years ago

@amolsontakke3576 Can you please share more details regarding the above endpoints so I can find the equivalent v3 and v4 endpoint and provide you as soon as possible earlier.

amolsontakke3576 commented 3 years ago

For Project Console

Request URL: https://api.lfcla.dev.platform.linuxfoundation.org/v1/salesforce/projects Request Method: GET Response : 

[
  {
    "name": "The Linux Foundation",
    "id": "a0941000002wBz9AAE",
    "description": "The Linux Foundation is dedicated to building sustainable ecosystems around open source projects to accelerate technology development and industry adoption.\nFounded in 2000, the Linux Foundation provides unparalleled support for open source communities through financial and intellectual resources, infrastructure, services, events, and training. Working together, the Linux Foundation and its projects form the most ambitious and successful investment in the creation of shared technology.",
    "logoUrl": "https://s3.amazonaws.com/cla-project-logo-dev/a0941000002wBz9AAE.png"
  },
  {
    "name": "Janusgraph",
    "id": "a0941000002wBybAAE",
    "description": "JanusGraph is a scalable graph database optimized for storing and querying graphs containing hundreds of billions of vertices and edges distributed across a multi-machine cluster.",
    "logoUrl": "https://s3.amazonaws.com/cla-project-logo-dev/a0941000002wBybAAE.png"
  }
]

Request URL: https://api.lfcla.dev.platform.linuxfoundation.org/v1/salesforce/project?id=a09410000182dD2AAI Request Method: GET Response : 

{
  "name": "Academy Software Foundation",
  "id": "a09410000182dD2AAI",
  "description": "The mission of the Academy Software Foundation (ASWF) is to increase the quality and quantity of contributions to the content creation industry\u2019s open source software base; to provide a neutral forum to coordinate cross-project efforts; to provide a common build and test infrastructure; and to provide individuals and organizations a clear path to participation in advancing our open source ecosystem.",
  "logoUrl": "https://s3.amazonaws.com/cla-project-logo-dev/a09410000182dD2AAI.png"
}

Request URL: https://api.lfcla.dev.platform.linuxfoundation.org/v1/project/486a553d-a1bd-4d78-b28f-914606c5f08f/configuration_orgs_and_repos Request Method: GET Response: 

{
  "orgs_and_repos": [
    {
      "auto_enabled": false,
      "branch_protection_enabled": false,
      "date_created": "2020-11-24T08:13:16.000000+0000",
      "date_modified": "2020-11-24T08:14:11.822523+0000",
      "organization_company_id": null,
      "organization_installation_id": "13150379",
      "organization_name": "TestBugsEasyCLAV2",
      "organization_name_lower": "testbugseasyclav2",
      "organization_project_id": null,
      "organization_sfid": "a0941000002wBz4AAE",
      "project_sfid": "a0941000002wBz4AAE",
      "version": "v1",
      "repositories": [
        {
          "repository_github_id": 315303902,
          "repository_name": "TestBugsEasyCLAV2/TestBugsEasyCLAV2-001",
          "repository_type": "github",
          "repository_url": "https://github.com/TestBugsEasyCLAV2/TestBugsEasyCLAV2-001",
          "enabled": true
        }
      ]
    }
],
"repositories": [
    {
      "date_created": "2020-09-03T11:02:17.000000+0000",
      "date_modified": "2020-10-22T11:53:18.000000+0000",
      "enabled": false,
      "note": "created on 2020-09-03T11:02:17Z. disabled on 2020-10-22T11:53:18Z",
      "project_sfid": "a092M00001IfVmKQAV",
      "repository_external_id": "276077439",
      "repository_id": "f2493e5b-b4d0-4e1a-a343-6553d6b22393",
      "repository_name": "lkkondepudi/repo1",
      "repository_organization_name": "lkkondepudi",
      "repository_project_id": "cd9548ee-0137-4641-a817-d6ae842e95b4",
      "repository_sfdc_id": "a0941000002wBz4AAE",
      "repository_type": "github",
      "repository_url": "https://github.com/lkkondepudi/repo1",
      "version": "v1"
    }
]
}
amolsontakke3576 commented 3 years ago

@rinkeshbhutwala Please check corporate console V1 API details in the below attachment https://app.zenhub.com/files/194341141/5d2a536d-c7c9-42bf-8174-67c893980e0b/download

amolsontakke3576 commented 3 years ago

@rinkeshbhutwala Please check contributor console V1 API details in the below attachment https://app.zenhub.com/files/194341141/7eeacdc9-d5f4-446b-a4c8-a63fdd4dc03d/download

rinkeshbhutwala commented 3 years ago

@dealako As discussed with @vinod-kadam on stand-up first try to fix the other bugs of v2 which are blocking for LFX and high priority so will work on this ticket later on because this is a big task for migrating all v1 endpoint to v3 go-lang endpoints.

dealako commented 2 years ago

Closing this old issue. We will re-evaluate at a later date if updates are required.