gruntwork-io / git-xargs

git-xargs is a command-line tool (CLI) for making updates across multiple Github repositories with a single command.
https://blog.gruntwork.io/introducing-git-xargs-an-open-source-tool-to-update-multiple-github-repos-753f9f3675ec
Apache License 2.0
935 stars 62 forks source link

If both --github-org and --repo are specified, git-xargs is still cloning all the repos in the org #50

Closed infraredgirl closed 2 years ago

infraredgirl commented 2 years ago

git-xargs version v0.0.11

Describe the bug If both --github-org and --repo are specified, git-xargs is still cloning all the repos in the org.

To Reproduce

$ git-xargs --branch-name=test-branch --github-org=git-xargs-test  --repo=test-repo-4  touch git-xargs-is-awesome.txt
[git-xargs] INFO[2021-09-21T15:43:26+02:00] git-xargs running...
[git-xargs] INFO[2021-09-21T15:43:27+02:00] Repository successfully processed             Repo name=test-repo-3
[git-xargs] INFO[2021-09-21T15:43:27+02:00] Repository successfully processed             Repo name=test-repo-2
[git-xargs] INFO[2021-09-21T15:43:27+02:00] Repository successfully processed             Repo name=test-repo-1

*****************************************************************
  GIT-XARGS RUN SUMMARY @ 2021-09-21 13:43:27.755367 +0000 UTC
  Runtime in seconds: 1
*****************************************************************

COMMAND SUPPLIED

[touch git-xargs-is-awesome.txt]

 REPOS SUCCESSFULLY FETCHED VIA GITHUB API
│────────────────│───────────────────────────────────────────────│
│ REPO NAME (4)  │ REPO URL                                      │
│────────────────│───────────────────────────────────────────────│
│ test-repo-1    │ https://github.com/git-xargs-test/test-repo-1 │
│ test-repo-2    │ https://github.com/git-xargs-test/test-repo-2 │
│ test-repo-3    │ https://github.com/git-xargs-test/test-repo-3 │
│ test-repo-4    │ https://github.com/git-xargs-test/test-repo-4 │
│────────────────│───────────────────────────────────────────────│

 ALL REPOS THAT WERE TARGETED FOR PROCESSING AFTER FILTERING MISSING / MALFORMED REPOS
│────────────────│───────────────────────────────────────────────│
│ REPO NAME (4)  │ REPO URL                                      │
│────────────────│───────────────────────────────────────────────│
│ test-repo-1    │ https://github.com/git-xargs-test/test-repo-1 │
│ test-repo-2    │ https://github.com/git-xargs-test/test-repo-2 │
│ test-repo-3    │ https://github.com/git-xargs-test/test-repo-3 │
│ test-repo-4    │ https://github.com/git-xargs-test/test-repo-4 │
│────────────────│───────────────────────────────────────────────│

 REPOS THAT WERE SUCCESSFULLY CLONED TO THE LOCAL FILESYSTEM
│─────────────│───────────────────────────────────────────────│
│ REPO NAME   │ REPO URL                                      │
│─────────────│───────────────────────────────────────────────│
│ test-repo-3 │ https://github.com/git-xargs-test/test-repo-3 │
│ test-repo-2 │ https://github.com/git-xargs-test/test-repo-2 │
│ test-repo-1 │ https://github.com/git-xargs-test/test-repo-1 │
│─────────────│───────────────────────────────────────────────│

 REPOS THAT WERE UNABLE TO BE CLONED TO THE LOCAL FILESYSTEM
│─────────────│───────────────────────────────────────────────│
│ REPO NAME   │ REPO URL                                      │
│─────────────│───────────────────────────────────────────────│
│ test-repo-4 │ https://github.com/git-xargs-test/test-repo-4 │
│─────────────│───────────────────────────────────────────────│

 REPOS THAT SHOWED NO FILE CHANGES TO THEIR WORKING DIRECTORY FOLLOWING COMMAND EXECUTION
│─────────────│───────────────────────────────────────────────│
│ REPO NAME   │ REPO URL                                      │
│─────────────│───────────────────────────────────────────────│
│ test-repo-3 │ https://github.com/git-xargs-test/test-repo-3 │
│ test-repo-2 │ https://github.com/git-xargs-test/test-repo-2 │
│ test-repo-1 │ https://github.com/git-xargs-test/test-repo-1 │
│─────────────│───────────────────────────────────────────────│

Expected behavior Only repos supplied with the --repo flag get cloned/processed, even if the --github-org flag is used as well.

zackproser commented 2 years ago

Hi @infraredgirl ,

Thanks for the report!

I was able to repro the issue you reported:

  1. The behavior of preferring --github-org over --repo is actually working as intended. It is documented here, but upon reflection I can see how it is confusing. Was your expectation that the org "git-xargs-test would have been used to specify the org, with the subsequent --repo flag serving as a filter to only operate on that specific repo within the specified org? git-xargs actually only operates on one mode at a time, with an order of preferences as stated in the above linked docs.
  2. The value you're passing to --repo in your example (test-repo-4) doesn't include the Github org prefix - so git-xargs considered this user-supplied repo invalid and discarded it
  3. There was no good error handling explaining this - but if you ran with --loglevel DEBUG you would have gotten a small, easy to miss line about your --repo value being ignored
  4. Therefore in the linked PR I added better error handling: a. If you supply only --repo values, but all are malformed, you'll get a more explicit error message explaining this now b. The "targeting mode" currently being used by git-xargs on any given run is now reported in the final print out - so that it's hopefully more clear what's happening and why

Please let me know if you'd like to discuss this further / go over it (if it's helpful for the ramp-up we're all doing) etc!

infraredgirl commented 2 years ago

Hey Zack, thanks for addressing this!

  1. The behavior of preferring --github-org over --repo is actually working as intended. It is documented here, but upon reflection I can see how it is confusing. Was your expectation that the org "git-xargs-test would have been used to specify the org, with the subsequent --repo flag serving as a filter to only operate on that specific repo within the specified org? git-xargs actually only operates on one mode at a time, with an order of preferences as stated in the above linked docs.

After I filed the issue, I did notice it's documented that the GH org takes precedence if present. I think intuitively I would still expect that, if more then one selection mode is used, the result should be their intersection. However, as this was the intended design, I think it's good enough if we document that (already done) and make explicit which mode is being used at runtime (added by your PR).

  1. The value you're passing to --repo in your example (test-repo-4) doesn't include the Github org prefix - so git-xargs considered this user-supplied repo invalid and discarded it

Interestingly enough, the reason that this repo was detected as malformed/invalid was not the lack of org prefix. It was because this was an empty/uninitiated repo that I created in GH for testing purposes. Just a data point for you - this is unlikely to come up in a real world usage.

  1. There was no good error handling explaining this - but if you ran with --loglevel DEBUG you would have gotten a small, easy to miss line about your --repo value being ignored
  2. Therefore in the linked PR I added better error handling: a. If you supply only --repo values, but all are malformed, you'll get a more explicit error message explaining this now b. The "targeting mode" currently being used by git-xargs on any given run is now reported in the final print out - so that it's hopefully more clear what's happening and why

Thanks - I will review the PR in question!

zackproser commented 2 years ago

Hey Zack, thanks for addressing this!

  1. The behavior of preferring --github-org over --repo is actually working as intended. It is documented here, but upon reflection I can see how it is confusing. Was your expectation that the org "git-xargs-test would have been used to specify the org, with the subsequent --repo flag serving as a filter to only operate on that specific repo within the specified org? git-xargs actually only operates on one mode at a time, with an order of preferences as stated in the above linked docs.

After I filed the issue, I did notice it's documented that the GH org takes precedence if present. I think intuitively I would still expect that, if more then one selection mode is used, the result should be their intersection. However, as this was the intended design, I think it's good enough if we document that (already done) and make explicit which mode is being used at runtime (added by your PR).

  1. The value you're passing to --repo in your example (test-repo-4) doesn't include the Github org prefix - so git-xargs considered this user-supplied repo invalid and discarded it

Interestingly enough, the reason that this repo was detected as malformed/invalid was not the lack of org prefix. It was because this was an empty/uninitiated repo that I created in GH for testing purposes. Just a data point for you - this is unlikely to come up in a real world usage.

  1. There was no good error handling explaining this - but if you ran with --loglevel DEBUG you would have gotten a small, easy to miss line about your --repo value being ignored
  2. Therefore in the linked PR I added better error handling: a. If you supply only --repo values, but all are malformed, you'll get a more explicit error message explaining this now b. The "targeting mode" currently being used by git-xargs on any given run is now reported in the final print out - so that it's hopefully more clear what's happening and why

Thanks - I will review the PR in question!

Ah understood. Thanks for the helpful context!