KeyError(hexsha) when cloning git://github.com/iwillig/fixcity.git

[slinkp]

I have installed the plugin snapshot schacon-hg-git-58dfc84ba193946730524845bb12f76b8ed87375.zip. It works for cloning some things, eg. I successfully cloned git://github.com/schacon/hg-git.git

But if I attempt to clone git://github.com/iwillig/fixcity.git I get this exception:

$ hg clone git://github.com/iwillig/fixcity.git
fetching from : origin
importing Hg objects into Git
Counting objects: 2013, done.
Compressing objects: 100% (1544/1544), done.
Total 2013 (delta 744), reused 1638 (delta 438)
importing Git objects into Hg
at:  0/21
** unknown exception encountered, details follow
** report bug details to http://www.selenic.com/mercurial/bts
** or mercurial@selenic.com
** Mercurial Distributed SCM (version 1.1.2)
** Extensions loaded: color, bookmarks, hg-git
Traceback (most recent call last):
  File "/usr/bin/hg", line 20, in <module>
    mercurial.dispatch.run()
  File "/var/lib/python-support/python2.6/mercurial/dispatch.py", line 20, in run
    sys.exit(dispatch(sys.argv[1:]))
  File "/var/lib/python-support/python2.6/mercurial/dispatch.py", line 29, in dispatch
    return _runcatch(u, args)
  File "/var/lib/python-support/python2.6/mercurial/dispatch.py", line 45, in _runcatch
    return _dispatch(ui, args)
  File "/var/lib/python-support/python2.6/mercurial/dispatch.py", line 367, in _dispatch
    ret = _runcommand(ui, options, cmd, d)
  File "/var/lib/python-support/python2.6/mercurial/dispatch.py", line 416, in _runcommand
    return checkargs()
  File "/var/lib/python-support/python2.6/mercurial/dispatch.py", line 376, in checkargs
    return cmdfunc()
  File "/var/lib/python-support/python2.6/mercurial/dispatch.py", line 361, in <lambda>
    d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
  File "/var/lib/python-support/python2.6/mercurial/util.py", line 715, in check
    return func(*args, **kwargs)
  File "/var/lib/python-support/python2.6/mercurial/commands.py", line 595, in clone
    update=not opts.get('noupdate'))
  File "/var/lib/python-support/python2.6/mercurial/hg.py", line 120, in clone
    src_repo = repository(ui, source)
  File "/var/lib/python-support/python2.6/mercurial/hg.py", line 61, in repository
    repo = _lookup(path).instance(ui, path, create)
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/gitrepo.py", line 17, in __init__
    git.fetch('origin')
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/git_handler.py", line 107, in fetch
    self.import_git_objects(remote_name, refs)
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/git_handler.py", line 644, in import_git_objects
    self.import_git_commit(commit)
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/git_handler.py", line 720, in import_git_commit
    files = self.git.get_files_changed(commit)
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/dulwich/repo.py", line 469, in get_files_changed
    all_changes.extend(filenames(otree, ptree, ''))
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/dulwich/repo.py", line 446, in filenames
    prefix + bname + '/'))
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/dulwich/repo.py", line 443, in filenames
    ctree = self.get_object(csha)
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/dulwich/repo.py", line 321, in get_object
    obj = self.object_store[sha]
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/dulwich/object_store.py", line 81, in __getitem__
    type, uncomp = self.get_raw(sha)
  File "/home/pw/Downloads/Apps/hg-git-plugin/hg-git/dulwich/object_store.py", line 210, in get_raw
    raise KeyError(hexsha)
KeyError: '69d9bee15b622fcd92b84ca352daf654f4f5fcd9'

[slinkp]

I just tried again with hg-git tip cloned from http://bitbucket.org/Scotty/hg-git/as of revision 177:88fdcb23294e ... same problem.

[slinkp]

I should have mentioned some version info: hg 1.1.2 python 2.6.2 ubuntu 9.04

[durin42]

I'm also seeing this on my local install with Dulwich 0.4.0 and hg 1.3.

[dmayle]

durin42, did you use some tool to import your git repository from a foreign scm? I've done some checking, and it looks like the problem is from dulwich not supporting empty commits...

[durin42]

I have no idea - this isn't my repository. I'll file an issue with dulwich about empty commits. Thanks.

[ianlewis]

I get this same error even though I don't have any empty commits in my repository.

[brainsik]

Been talking to the dulwich maintainer on launchpad and they believe this is a problem with how hg-git retrieves submodules:

https://bugs.launchpad.net/dulwich/+bug/480409/comments/6

For testing, here is a simple repository (2 files / 13 commits) also causing the traceback:

git://github.com/dydx/Project.git

I found it just by cloning random repos from github. Here are some more:

git://github.com/wildabeast/phonegap-symbian.qt.git git://github.com/pietern/jspec-async.git git://github.com/tarcisio/pesquisa_vagas.git

[DarthNerdus]

I too have seen this error a few times...

Here are a few more examples: git://github.com/rentzsch/mogenerator.git git://github.com/ccgus/jstalk.git git://github.com/Caged/gitnub.git

Mercurial 1.4.2 dulwich 0.4.1 hg-git directly from repo as of January 24th.

[thomie]

I had a similar keyerror and fixed it after reading this issue. Cloning some of the repositories mentioned above still fails though.

[brainsik]

Unfortunately, this didn't fix any of the repos I tried. :-(

[thomie]

So it seems the problem is indeed with git repositories that contain submodules.

For example the 9th commit in github.com/dydx/Project introduces a submodule. Cloning it with hg-git up to that commit (first git clone it, then git reset --hard HEAD~5, and manually remove the refs in .git/packed-refs referring to origin) works fine.

Since hg-git doesn't support git submodules, all I could think of for now was to catch the KeyError, and print a nicer abort message.

[brainsik]

Alright, I have patched hg-git (cc0b7fa90d46beb2d0fc21d5d9b7ea9076fbb7d7) to not import submodules. It was reviewed by one of the mercurial crew devs and is being looked at now by durin42. It's a small change. Feel free to pull and grab those repos that were giving you problems.

[slinkp]

Works for me now, thanks!

[durin42]

Fixed.