Translate git sha1 to/from hg sha1

[jedbrown]

Interacting with hg users on mailing lists or web trackers would be greatly assisted by a way to translate commit identity. Ideally, git rev-parse could be taught to recognize something like hg/<sha1> and have a way to include hg's sha1 in git log.

[dusty-phillips]

This is a good idea, although my

git-rev-parse doesn't have any hook mechanism for specifying revisions, so this would require a change inside git. I'm not ruling it out if the patch were to be accepted upstream.

I think adding a "HG SHA:" line to the --pretty format would also require adding some kind of hook to git-log, but it's certainly doable.

[jedbrown]

Are you on the git mailing list? Although I'd expect upstream to be receptive, implementation of such hooks will probably require some discussion on their list first.

[xentac]

I'm sitting beside Dusty in an airport right now. I've figured out a way to store backrefs from hg sha1s to git sha1s by storing each one as a ref (under ref/hg/...). This will make any git tool able to lookup hg/thehgsha1. I'm going to try implementing it now, but I can't currently think of a way to make git log backreference this information.

[sschuberth]

How about just storing hg sha1s as git notes in git commits?

[xentac]

That would make it possible to look at a git commit log and find the corresponding hg hash, but it wouldn't help any internal git tools to refer to hg commits.

[dusty-phillips]

I looked into this, but I don't think there's any way to search or query notes.

[sschuberth]

@buchuki I'm not sure what exactly you're looking for, but there is git notes list and you can also git grep on refs/notes/commits.

[fingolfin]

On a tangential note: It might be worth considering to rewrite gitifyhg to use hg sha1s to track commits instead of the (only locally valid) rev ids. The primary point for this would be to make things more robust; the rev ids can change (e.g. if one has to strip some commits "in the middle", due to a partially failed push, see #6). The drawback would be that some optimizations would be impossible, such as revs = xrange(tip, head.rev() + 1). The latter might be a deal breaker for huge hg repositories, though. Then again, the hg client itself must deal with that somehow, so I wonder if there isn't a way to re-implement this code using SHA1s w/o sacrificing too much efficiency.

[dusty-phillips]

I would very much like to see the marks tracked as sha1s instead of integers. I'm not sure about the efficiency concerns, though. I know it takes FOREVER to clone pypy, and I'd like to see that optimized, but I think the serial nature of git-remote might preclude it.

PS: bonus points for 'tangential' ;)

[fingolfin]

As a temporary way to solve this would be to add a gitifyhg executable (resp. make the existing executable do something different depending under which name it is called, and install / symlink it to both PREFIX/bin/git-remote-gitifyhg and PREFIX/bin/gitifyhg.

The gitifyhg command then could have subcommands that allow translating hg hashes to git hashes and vice versa, by looking at the mark files. It could also do other things, like offering a "help" command, allowing to tweak certain gitifyhhg configuration etc.

A variant of would be to install this as a git subcommand, i.e. as PREFIX/bin/git-gitifyhg (or just "git-hg" :). And / or other subcommands for translating hashes... something like

git gitifyhg hg-ref-to-git COMMITISH git gitifyhg git-ref-to-hg COMMITISH

or so. It's not ideally, but reasonably easy to implement and useful... i.e. pragmatic, until somebody comes up with something better :-).

[fingolfin]

One step to something better probably is to follow sschuberth's idea and use git notes to store the hg sha1 on the corresponding git commits. Indeed, I believe I recall that the guys behind the msysgit git-remote-hg (Sverre Rabbelier and Johannes Schindelin) also want(ed) to use them.

They are quite nice: E.g. one can tell git log to display notes.

Indeed, one could even store these special "notes" not in the usual refs/notes/commits`` namespace, but rather in a custom namespace likerefs/notes/hg`, and then add a subcommandgitifyhg log $*which would be an alias for``git log --notes=hg $*```. Additionally, users can set core.notesRef or notes.displayRef to enable showing these automatically with "git log", too.

The same remarks (and settings ;) apply to e.g. "git show". So this gives a great way to map git SHA1s to hg.

For the other way around, one can of course grep through refs/notes/hg, but that is not super user friendly, so having a git (or gitifyhg) subcommand for that would still be nice, even if it is just a simple alias to git grep or so.

[dusty-phillips]

I was thinking of having the script edit .git/config to point a bunch of aliases at gitifyhg commands, when they become necessary, but a subcommand might be better.

[fingolfin]

I'd be wary of changing the user's .git/config. And when would you do it anyway? We don't want multiple entries to end up in there... so you might check if a given alias already exists. but what if you later improve the alias -- then existing clones would not benefit from the improvement. Also, aliases are rather limited in the end... Finally, changing the config always bears the risk of conflicting with some other changes the user made there, and that could easy upset some folks greatly...

so to me a separate gitifyhg command (resp. git subcommand) (or multiple) seems safer... just saying. :-)

[dusty-phillips]

I would have done it at the time of the clone, so before the user has had a chance to edit it. However, I cannot find anything to recommend this approach over a subcommand. It might be fewer characters to type, but if someone's anal about that, they can write their own alias!

[jedbrown]

@fingolfin You won't get multiple entries because you edit with git config, but I dislike the security implications of editing .git/config. Also, since people can git remote add gitifyhg::..., you don't always get a fresh clone. I prefer explicit sub-commands. FWIW, there is a precedent for git svn find-rev SVN_REV. For going the other way, I also think that notes are the right way to implement. This thread concluded the same for git-svn, but it seems like people consider that project to be "good enough" so there's no active development:

http://thread.gmane.org/gmane.comp.version-control.git/174657

This thread discusses using tags to go the other way:

http://thread.gmane.org/gmane.comp.version-control.git/182158

[fingolfin]

@jedbrown git explicitly supports multi keys, and you can get those when using git config, too, but yeah, it is trivial to avoid creating them by accident when using git config (and one should never attempt to modify that config file by other means). But ah, I should stop the nitpicking, because it seems we actually agree overall. Good. :-) Now all that is missing is that somebody implements this :grin:

So, (assuming buchuki agrees), somebody could whip up a first stab at a gitifyg subcommand, doing nothing for now except providing a place for adding actually sub-subcommands, And perhaps just call it "hg"... I mean, sure we might clash with other projects, but since none of the has a "git hg" yet, well, let them worry about not clashing with gitifyhg ;-). So one can do stuff like "git hg find-rev" or "git hg log" or so... But hey, changing the name of the subcommand is easy enough to do.

The notes stuff is another project... looking at the git-fast-import docs, there is a notemodify command which can be put into the fastimport stream to add notes to the commit, even under a custom ref. Unless I am missing something, this means it should be super easy to add those notes in... Neat.

[dusty-phillips]

I can rubberstamp all that. I'd also like to see #16 implemented as part of this. Great discussion guys!

[jedbrown]

On naming, there is a git-remote-hg in the git repository, though it fails a lot more tests than gitifyhg at this point, so it could presumably be replaced once gitifyhg is more stable.

[fingolfin]

Yeah, felipe's remote-hg is currently in git.git/contrib. Hence I am not suggesting (for now, at least) to rename git-remote-gitifyhg. However, there is no git-hg in git.git right now, so that could be used (to implement the git subcommands we were talking about). But of course git-gitifyhg would also be usable, albeit longer to type and somewhat redundantly named... git hg log vs. git gitifyhg log... but it is a) easy to rename it either way, and b) easy for a user to add an alias hg=gitifyhg :-).

[jedbrown]

Does anyone know how to make notes propagate through a subsequent clone besides git clone --mirror?

[jedbrown]

Note that you can use globs for the notes argument. Thus git log --notes=hg* <committish> combined with git log --notes=hg* --grep=<hg-sha1> provide two-way lookup. The user can make aliases for these. I'm still looking for a way to automatically merge into refs/notes/hg, but it does not seem possible using the fast-import protocol.

[jedbrown]

More convenient trick:

git symbolic-ref refs/notes/hg refs/notes/hg-SHA1
git config core.notesRef refs/notes/hg

Now git log will always show you the available notes and git log --grep=HGSHA1 will locate the git commit. This doesn't merge notes from multiple remotes, but it's probably okay for most use cases where there will be one publishing clone that is a superset of the others.

[dusty-phillips]

Jed, I've lost track of this issue. Can you describe what needs to be done to mark it as closed? Perhaps the "next phase" should be a separate issue?

[jedbrown]

I'm satisfied with the method described above that you added to the README. It's well-defined enough to be used in scripts and grepping the notes is adequately fast. For a more integrated interface, I would suggest adding git gitifyhg-revparse HGSHA1 that would just grep the notes properly without needing user configuration, but the current interface just isn't causing me any trouble. In any case, that would reasonably be another issue so I'll close this one.

[dusty-phillips]

Sweet, closed issues are so pretty. Great work on this one!