hasura / smooth-checkout-buildkite-plugin

All the things you need during a Buildkite checkout :butter: :kite:
Apache License 2.0
14 stars 12 forks source link

Add `depth` parameter to aid in shallow clones #27

Closed staticfloat closed 1 year ago

staticfloat commented 2 years ago

This adds a depth parameter, which can be used to pass --depth arguments to git clone, vastly reducing network traffic when a full clone is not necessary.

TSMMark commented 2 years ago

I would love to see this. Would maintainers accept a PR?

TSMMark commented 2 years ago

What I'm dying to be able to do is actually git diff between 2 shallow clones depth=1 between current branch and main branch.

This looks really a promising solution: EDIT: I found a better way, posted in https://github.com/hasura/smooth-checkout-buildkite-plugin/pull/27#issuecomment-1198280071

git init .                                               # Create an empty repository
git remote add origin git@github.com:lodash/lodash.git   # Specify the remote repository

git checkout -b base                                     # Create a branch for our base state

git fetch origin --depth 1 4.0.0                         # Fetch the single commit for the base of our comparison
git reset --hard FETCH_HEAD                              # Point the local master to the commit we just fetched

git checkout -b target                                   # Create a branch for our target state

git fetch origin --depth 1 master                        # Fetch the single commit for the target of our comparison
git reset --hard FETCH_HEAD                              # Point the local target to the commit we just fetched

git diff --name-only base target                         # Print a list of all files changed between the two commits

From here https://dev.to/eoinsha/find-changes-between-two-git-commits-without-cloning-4kkp

And I'm hoping to be able to use this plugin to do the shallow cloning

TSMMark commented 2 years ago

I've been using this fork of the plugin for a few days and it works great!

I also found a nice technique to deepen a shallow clone until the merge base commit is found. Posting it here in case it helps someone else:

#!/bin/bash
set -euo pipefail

# From: https://stackoverflow.com/a/56113247/2696867
echo "--- git fetching shallow merge base"

# TODO: Consider using PR base branch $BUILDKITE_PULL_REQUEST_BASE_BRANCH, and default to master if no PR.
echo "Fetching commits until we find the merge-base / fork-point between current commit and master"
while [ -z $( git merge-base master $BUILDKITE_COMMIT ) ]; do
  echo "git fetch --deepen=50 origin master $BUILDKITE_COMMIT"
  git fetch --deepen=50 origin master $BUILDKITE_COMMIT
done

echo "Done."

This allows you to have a shallow clone but still have it go as deep as you need to be able to git diff between current branch and base branch:

git --no-pager diff master... --name-only
imperfect-fourth commented 2 years ago

Hey @staticfloat, thanks for this contribution! I'm taking a look at this now.

imperfect-fourth commented 1 year ago

@TSMMark @staticfloat with v4.1.1 you can use clone_flags field to pass whatever flags you want to the git clone command. It can also be used to do a shallow clone as in this example .

imperfect-fourth commented 1 year ago

Closing as it's no longer required