gc breaks gitifyhg

[pv]

Hi,

Running the following causes problems:

git clone gitifyhg::https://bitbucket.org/pypy/pypy
cd pypy
git reflog expire --expire=0 --all
git prune
git repack -f -a -d
git gc --prune=0

After a few days when the repo got some updates:

$ git fetch
error: unable to find 0c14a35e49fa285d4132297d47cb83ff2a039854
fatal: object not found: 0c14a35e49fa285d4132297d47cb83ff2a039854
fast-import: dumping crash report to .git/fast_import_crash_4268
fatal: Error while running fast-import
Traceback (most recent call last):
  File "/usr/local/bin/git-remote-gitifyhg", line 9, in <module>
    load_entry_point('gitifyhg==0.8.2', 'console_scripts', 'git-remote-gitifyhg')()
  File "/usr/local/stow/python-easy-install/lib/python2.7/site-packages/gitifyhg-0.8.2-py2.7.egg/gitifyhg/gitifyhg.py", line 342, in main
    HGRemote(*[x.decode('utf-8') for x in args]).process()
  File "/usr/local/stow/python-easy-install/lib/python2.7/site-packages/gitifyhg-0.8.2-py2.7.egg/gitifyhg/gitifyhg.py", line 202, in process
    getattr(self, 'do_%s' % command)(parser)
  File "/usr/local/stow/python-easy-install/lib/python2.7/site-packages/gitifyhg-0.8.2-py2.7.egg/gitifyhg/gitifyhg.py", line 305, in do_import
    HGImporter(self, parser).process()
  File "/usr/local/stow/python-easy-install/lib/python2.7/site-packages/gitifyhg-0.8.2-py2.7.egg/gitifyhg/hgimporter.py", line 99, in process
12:08 (master)pauli@Obelisk:~/prj/external/pypy$     self.process_ref(name, reftype, head)
  File "/usr/local/stow/python-easy-install/lib/python2.7/site-packages/gitifyhg-0.8.2-py2.7.egg/gitifyhg/hgimporter.py", line 183, in process_ref
    output(data)
  File "/usr/local/stow/python-easy-install/lib/python2.7/site-packages/gitifyhg-0.8.2-py2.7.egg/gitifyhg/util.py", line 51, in output
    print >> actual_stdout, msg
IOError: [Errno 32] Broken pipe

The same issue occurs with git-remote-hg from git/git/contrb, btw.

Due to dropping anonymous heads? I'd guess the same issue will come about even without running the gc if you wait a couple of months so that git auto-expires unreachable commits?

[charleseff]

+1, I'm also getting this.

[fingolfin]

That sounds really bad :-/.

BTW, you mention that it also affects git-remote-hg from git.git/contrib -- you may wish to inform its author about this issue, too (e.g. by filing a report here: https://github.com/felipec/git/issues)

[fingolfin]

I can reliably reproduce this issue with a local clone of the pygments repository (which is much smaller than the pypy repository, which is helpful). This does it:

hg clone http://bitbucket.org/birkenfeld/pygments-main
git clone hg::pygments-main pygments.git
cp -r pygments.git pygments.git.bak  # makes it easier to debug the issue
cd pygments.git
git reflog expire --expire=0 --all
git prune
git repack -f -a -d
git gc --prune=0

After this, if I do a pull (even without any remote changes, I get an error like this:

$ git pull
fatal: object not found: 21a43f1a47819533ebc1144382316104f460cf77
fast-import: dumping crash report to .git/fast_import_crash_13549
fatal: Error while running fast-import

Going to the backup copy of the clone I made, it is easy to find out what the missing object is:

$ git show --notes=hg* 21a43f1a47819533ebc1144382316104f460cf77
commit 21a43f1a47819533ebc1144382316104f460cf77
Author: Georg Brandl <georg@python.org>
Date:   2010-09-12 19:47:49 +0200

    Close inactive branch.

Notes (hg):
    ab1b6651b3c730c5e36dd90f6835926cf8ecbb1d

Aha! So the fact that we don't create refs for those branches is causing the commits on them to be garbage collected. BTW, for reference, here is how the above commit look on the Mercurial side:

$ hg show ab1b6651b3c730c5e36dd90f6835926cf8ecbb1d
changeset:   1080:ab1b6651b3c7
branch:      perl-CHECK-INIT-block-highlighting
parent:      1024:ef8e4ba205d6
user:        Georg Brandl <georg@python.org>
date:        Sun Sep 12 19:47:49 2010 +0200
description:
Close inactive branch.

Now, one fix for this would be add refs for all closed branches (kind of ugly, though). But even with this, we'd run into an issue if a mercurial branch has multiple heads...