Bug Report: API returns inaccurate results due to Access Token constraints

chriscerk commented 4 months ago

I'm formally logging here as a bug since the API not returning accurate data is not expected.

(the support email thread got logged in the below discussion item, but I think this should be treated as a bug - not a new feature)

Related items

The current work-a-round for getting accurate data for an access token is quite lengthy:

manually sign in as the user which the access token was provisioned
click the re-sync repo's button via the UI

This isn't acceptable for a programatic use of the API.

Describe the bug A clear and concise description of what the bug is.

Calling the codecov API via an access token does not return accurate data.

Environment (please complete the following information):

System: any use of an access token

To Reproduce Steps to reproduce the behavior:

Have a large number of repositories
Create an access token and call the active repos via a script
Create new repositories
Note that new repositories and other calls relying on those new repositories will fail

Expected behavior A clear and concise description of what you expected to happen.

We are able to call the codecov API and get accurate data.

Either via

[API] Create endpoint to trigger a resync
or using some other authentication method not based on these access tokens which have constraints

claym-ih commented 4 months ago

👍

chriscerk commented 3 months ago

Any updates on this?

(even if it is a "no we won't support this" or this will re-prioritized in MM/yyyy)

chriscerk commented 1 month ago

Another bump - any updates?

drazisil-codecov commented 1 month ago

@aj-codecov can we get an update, please?

aj-codecov commented 1 month ago

@chriscerk Hey! Sorry for the lack of movement here. How long before the API seems to be returning all of the correct info after new repo creation? Never? Are we talking roughly 3K+ repos? It almost sounds to me like it could be that we're just slow to sync the new updates and might need to do some optimization there if we eventually sync the updates. Does this sound right?

chriscerk commented 1 month ago

Context

How long before the API seems to be returning all of the correct info after new repo creation?

It has taken up to 24h previously.

Are we talking roughly 3K+ repos?

For our Github Org:

active: 434 repositories
archived: 334 repositories

We are not at that scale of 3K+.

It almost sounds to me like it could be that we're just slow to sync the new updates and might need to do some optimization there if we eventually sync the updates.

There is an open idea to have a customer-facing endpoint to trigger the re-sync.

CodeCov UI already has this: (just not exposed for customers yet)

POST https://api.codecov.io/graphql/github

    mutation SyncData {
        syncWithGitProvider {
          me {
            isSyncing: isSyncingWithGitProvider
          }
          error {
            __typename
          }
        }
      }

resulting in:

{"data": {"syncWithGitProvider": {"me": {"isSyncing": true}, "error": null}}}

Proposal

Support programatic control for customers to sync source control provider metadata and view the current sync status.

`PUT https://api.codecov.io/api/v2/PROVIDER/OWNER_USERNAME/sync`

Expose endpoint which proxies GraphQL mutation SyncData functionality and response.

`GET https://api.codecov.io/api/v2/PROVIDER/OWNER_USERNAME/sync`

Expose endpoint which proxies GraphQL query IsSyncing functionality and response.

chriscerk commented 1 month ago

⬆️ @aj-codecov

forgot to tag you

aj-codecov commented 1 month ago

@chriscerk I appreciate the clarification, let me chat with my engineering team and see if I can't get you an answer re: timeline or some sort of path forward

aj-codecov commented 1 month ago

@chriscerk I spoke with a couple of our engineers about this and they created this issue: https://github.com/codecov/engineering-team/issues/2102 - the solution here is to listen for the repository webhook for repo creation, which we currently don't do, so that we populate the new repo immediately. Feel free to weigh in here (or on that issue) if you disagree with the solution, but that feels like it should solve things without you having to hit an extra endpoint - in my opinion, this should just work the way you originally expected.

As far as timeline, we're at the end of our quarter right now so we're jam packed and trying to wrap up some larger items, we should be able to tackle this in the first few weeks of August though.

chriscerk commented 1 month ago

@aj-codecov Awesome!

Efficiency-wise that is definitely much cleaner on your end.

I was just suggesting the prior solution as a programatic approach to trigger the sync repo's which already existed.

we should be able to tackle this in the first few weeks of August though.

woohoo! 🎉

codecov / feedback