Prune stale remote branches from gitlab

[rdzah]

Hi, we use gitlab-mirrors to mirror several repos from github to our gitlab install.

Our issue is that when we delete remote branches from our github repo, the corresponding mirrored branch is not removed from gitlab (though it is removed from the gitlab-mirrors version of the repo).

Thus, we end up with an unwieldy number of stale branches clogging up the gitlab ui.

Any ideas? Gitlab can't have gitlab-mirrors as a remote, so git fetch -p won't work...

Thanks for the great project!

[samrocketman]

This is a good point. I've been conflicted about how to handle pruning. On one hand I have this commit blob: d4d3934981475b6ce201dbb2ff47bdace2504ddf. In that scenario I'm actually getting rid of pruning altogether but it makes sense to clean up things that are remote tracking.

One could use the combination of git commands show-ref and ls-remote to compare local references and remote references. Then a git push --delete could be issued on a ref that doesn't exist in the local show-ref but does exist remotely in ls-remote.

An example could be...

git remote prune origin
#remove gitlab remote branches and tags that don't exist locally
for x in $(git ls-remote --heads --tags gitlab | awk '{print $2}'); do
  if ! git show-ref 2> /dev/null | grep "${x}" &> /dev/null; then
    #delete the remote ref with following command
    git push gitlab :${x}
  fi
done

Note the above code is untested. I wrote it purely by looking at the ls-remote and show-ref man pages. It's an idea though. I'm curious to hear your thoughts both on this comment as well as that commit blob where I removed pruning from origin entirely.

[samrocketman]

Btw @rdzah which version of gitlab-mirrors are you using? Also, which version of GitLab are you mirroring against (and did you install via official system packages or from source). I've been getting reports recently about gitlab-mirrors not being able to mirror against later versions of GitLab but my computer environment is out of commission at the moment so I haven't really had the capability to test. Please let me know.

[rdzah]

Your pruning script works great! I have it running in a cron job for each of our github repos we mirror to gitlab.

Re: prune origin, perhaps pruning of upstream origin branches/tags from the mirror along with pruning of the downstream gitlab target could both be toggles?

We’re using gitlab version 6.9.2, installed from source, and gitlab-mirrors 4.8 from master. Re: newer gitlab incompatibilities, that makes me fear to update gitlab only to find a dead mirror…

Thanks

[samrocketman]

@rdzah the incompatibilities reported purely come from adding a project. If your projects already exist then it wouldn't be an issue. gitlab-mirrors purely uses git to do the mirroring. However, it interacts with the GitLab API to create projects which is where the reports come in being a hiccup. I'll have to test it. New harddrive came in today so I'll be able to start my recovery :godmode: .

Good idea on making it an option. I think I'll add that as an option. I'll mark this ticket as an enhancement. Nice to hear my makeshift script works xD.

[samrocketman]

@rdzah You're going to laugh at me but... after reading the man pages and seeing one of my forks... specifically https://github.com/margnus1/gitlab-mirrors/commit/f59a5aefe135cb6784b7c55c0315109156f4fbe7. I've recently learned about git push --prune gitlab. git push has a --prune option which operates similar to git remote prune which essentially does the exact same thing as that overly long script I gave you... So in short all of that code can be modified as...

git remote prune origin
git push --prune gitlab

So there ya go haha. Git devs already thought of that. Of course they did because it's git.