concourse-bot
commented
7 years ago Hi there!
We use Pivotal Tracker to provide visibility into what our team is working on. A story for this issue has been automatically created.
The current status is as follows:
- [ ] #134119039 PROPOSAL: Performance optimizations by checking Github API to avoid clones which would be no-ops
This comment, as well as the labels on the issue, will be automatically updated as the status in Tracker changes.
I'm considering modifying the git resource with an optional feature to be less resource intensive by making it only perform clones (even shallow) when there are new commits to clone. This could be done in a github-specific way by using this endpoint: https://developer.github.com/v3/repos/commits/#list-commits-on-a-repository - and also using conditional requests (https://developer.github.com/v3/#conditional-requests) to avoid hitting the github quota.
This would be useful when there are many instances pipelines having checking/triggering git resources and thus generating much github network traffic and requests via clones, such as tends to happen with multi-pipeline approaches like https://github.com/robdimsdale/concourse-pipeline-resource or https://github.com/pivotaltracker/concourse-branch-manager