gruntwork-io / git-xargs

git-xargs is a command-line tool (CLI) for making updates across multiple Github repositories with a single command.
https://blog.gruntwork.io/introducing-git-xargs-an-open-source-tool-to-update-multiple-github-repos-753f9f3675ec
Apache License 2.0
944 stars 62 forks source link

Simplify re-running on repos that failed #108

Open jdimatteo opened 1 year ago

jdimatteo commented 1 year ago

Describe the solution you'd like

Optionally write all the failed to update repos to a .txt file in the format that git-xargs --repos accepts.

Describe alternatives you've considered

In the process of updating about 2500 repos I ran into multiple errors and had to re-run git-xargs multiple times, and the process looked like this:

1) run git-xargs redirecting stdout / stderr to a file 2) copy the tables following text like "Repos whose initial Pull Request failed to be created due to GitHub rate limits" or "Repos that were unable to be cloned to the local filesystem" etc. to a txt file 3) run a vim macro to remove the table formatting (e.g. ------) so that it is in a format that git-xargs --repos can accept 4) re-run git-xargs with --repos arg pointing to that manually created file, and repeat steps 1-4 until all repos successfully processed or intentionally skipped

Ideally there wouldn't be any errors, but with so many repos that might be impractical, and the errors could also be user-error based on the command passed to git-xargs. A more efficient user experience might be the following:

a) run git-xargs redirecting stdout / stderr to a file and the option --failed-repo-output-file /tmp/failed_to_update_repos.txt b) review stdout / stderr and remove any repos from /tmp/failed_to_update_repos.txt that you intentionally don't want to retry c) run git-xargs with --repos /tmp/failed_to_update_repos.txt and --failed-repo-output-file /tmp/failed_to_update_repos2.txt etc. (and it might be good to support --repos and --failed-repo-output-file pointing to the same file)

diogokiss commented 1 year ago

I'd even consider this request something similar to the ability to be idempotent. It should be able to pick up from where it stopped in case of errors.