These changes fix some sneaky concurrency bugs that were manifesting in various ways, either with PRs failing to be opened correctly or occasional opaque waitgroup errors that are hard to replicate because they're all timing related.
The core issue was mishandling of the waitgroup that was being created to orchestrate the work of operating on repositories and handling pull request opening with awareness of the GitHub API rate limits.
These changes simplify the waitgroup logic:
Uses the stdlib's sync.WaitGroup
Only call waitgroup.Add once at the beginning of processing and set its length to the total number of repositories to be processed
Plumb the waitgroup through the entire workflow, including the pull request retry workflow that was introduced in #67
These changes make it much simpler to reason about the concurrent logic and in local testing they resolve the issues observed in #77 and #74.
These changes fix some sneaky concurrency bugs that were manifesting in various ways, either with PRs failing to be opened correctly or occasional opaque waitgroup errors that are hard to replicate because they're all timing related.
The core issue was mishandling of the waitgroup that was being created to orchestrate the work of operating on repositories and handling pull request opening with awareness of the GitHub API rate limits.
These changes simplify the waitgroup logic:
These changes make it much simpler to reason about the concurrent logic and in local testing they resolve the issues observed in #77 and #74.
Description
Fixes #77 and #74.
TODOs
Read the Gruntwork contribution guidelines.
Release Notes (draft)
Added / Removed / Updated [X].
Migration Guide
The
--max-concurrent-repos
flag has been removed. Please update any earlier git-xargs invocations / scripts that use that flag to remove it.