gruntwork-io / git-xargs

git-xargs is a command-line tool (CLI) for making updates across multiple Github repositories with a single command.
https://blog.gruntwork.io/introducing-git-xargs-an-open-source-tool-to-update-multiple-github-repos-753f9f3675ec
Apache License 2.0
933 stars 63 forks source link

Using `--github-org` only returns 25 repositories #87

Closed BastienM closed 2 years ago

BastienM commented 2 years ago

Last time I used git-xargs was in May of this year, with the v0.0.14. And everything was fine. I recently upgraded all my brew packages and nothing seems to works with the v0.0.16.

Describe the bug

When using --github-org to scan my company's org, git-xargs resolves only 25 of the few hundreds we have.

$ git-xargs --version
git-xargs version v0.0.16

$ git-xargs --loglevel DEBUG \
            --pull-request-title "Some title" \
            --commit-message "Doing things" \
            --branch-name "fix/my-things" \
            --max-concurrent-repos 1 \
            --github-org <ORG> --skip-archived-repos --dry-run touch helloworld.txt

[git-xargs] INFO[2022-07-20T16:56:14+02:00] git-xargs running...                         
[git-xargs] INFO[2022-07-20T16:56:14+02:00] Dry run setting enabled. No local branches will be pushed and no PRs will be opened in Github 
[git-xargs] DEBU[2022-07-20T16:56:15+02:00] Skipping archived repository                  Name=<ORG>/***
[git-xargs] DEBU[2022-07-20T16:56:15+02:00] Skipping archived repository                  Name=<ORG>/***
[git-xargs] DEBU[2022-07-20T16:56:15+02:00] Skipping archived repository                  Name=<ORG>/***
[git-xargs] DEBU[2022-07-20T16:56:15+02:00] Fetched repos from Github organization: <ORG>  Repo count=25
[git-xargs] DEBU[2022-07-20T16:56:15+02:00] Using Github org: <ORG> as source of repositories. Paging through Github API for repos. 
[...]
*****************************************************************
  GIT-XARGS RUN SUMMARY @ 2022-07-20 15:07:32.722381 +0000 UTC
  Runtime in seconds: 51
*****************************************************************

COMMAND SUPPLIED

[touch helloworld.txt]

REPO SELECTION METHOD USED FOR THIS RUN - (see README.md for more information)

github-org

 REPOS SUCCESSFULLY FETCHED VIA GITHUB API
| REPO NAME (22)          | REPO URL                                          |
|-------------------------|---------------------------------------------------|

To Reproduce

Expected behavior

For git-xargs to actually follow up on the API pagination.

Nice to have

Additional context

Edit: ℹ️ This one was coming from an improper permission set on the token I also tried supplying a list via --repos but every call ends up in a 404.

$ cat repos.txt | wc -l
      81
$ git-xargs --loglevel DEBUG \
            --pull-request-title "Some title" \
            --commit-message "Doing things" \
            --branch-name "fix/my-things" \
            --max-concurrent-repos 1 \
            --repos repos.txt --skip-archived-repos --dry-run touch helloworld.txt
[git-xargs] INFO[2022-07-20T17:17:02+02:00] git-xargs running...                         
[git-xargs] INFO[2022-07-20T17:17:02+02:00] Dry run setting enabled. No local branches will be pushed and no PRs will be opened in Github 
[...]
[git-xargs] DEBU[2022-07-20T17:17:02+02:00] Looking up filename provided repo             Name=documents Organization=<ORG>
[git-xargs] DEBU[2022-07-20T17:17:04+02:00] error getting single repo                     AllowedRepoName=documents AllowedRepoOwner=PayFit Error="GET https://api.github.com/repos/<ORG>/documents: 404 Not Found []" Response Status Code=404
[...]

$ gh repo view <ORG>/documents 
<ORG>/documents

[SHOWS THE README]
zackproser commented 2 years ago

Hi, it's possible this is a duplicate of #77 - could you please give #85 a shot and see if that branch resolves your issue?

Thanks!

BastienM commented 2 years ago

Hi @zackproser,

I tested master and v0.0.14 alternatively with the same results. Which led me to have a look at the token's permissions, and the issue lied there. Bad configuration from my end.

With a correctly setup token everything works fine 👌

Apologies for the noise 🙇

zackproser commented 2 years ago

Roger - thanks for closing the loop and no worries :+1: