search

dusty-phillips / gitifyhg

Tools for using git as a client to mercurial repositories
GNU General Public License v3.0
62 stars 17 forks source link

Check mercurial versions, track compatibility #20

Closed fingolfin closed 11 years ago

fingolfin commented 11 years ago

One major problem troubling all remote-hg implementations I have seen so far is that that the hg python "API" is not really a documented API -- it changes with each release, and sometimes in incompatible ways. This is also documented in the gitifyhg documentation

As a result, though, a new release can break these remote helpers. And on the other hand, not everybody is running mercurial 2.4.x yet, many people are stuck with older versions installed by their distros. And these older versions may not be compatible.

Hence, I think it would be important to figure out with which versions of hg gitifyhg is really compatible, and document that. Moreover, as change are made, this should be re-tested.

A good way to do that would be to have a machine with multiple hg versions installed in parallel -- say the latest from 2.4.x, 2.3.x, 2.2.x etc. and/or the versions of mercurial used by some major linux distros (e.g. the ones used by the latest debian and ubuntu LTS releases). And then simply run the gitifyhg test suite on each.

I know this can easily be a lot of work, but I think it would be well worth it to set this up once, and then just let it run. Indeed, I wonder if https://travis-ci.org/ could be used for that. It sounds perfect for this kind of job, and integrates nicely with github. See also

dusty-phillips commented 11 years ago

It would probably be quite trivial to test this with tox.

fingolfin commented 11 years ago

Tox sounds indeed very useful for this, and sounds to me as if it could make it a lot more convenient to test across multiple mercurial versions etc.. Reading up a bit on tox, pip, virtualenv etc. made me realize that doing such multi version testing might be easier to get going than I thought. Nice. (As you can see, I am not really a python expert, but I am happy to learn).

That said, I still think it might be nice to use travis, too -- together with or parallel to tox. -- indeed, I would hope that it is easy to combine the two... One nice thing about travis is that the tests are automatically run on their served, based on certain triggers. E.g. automatically on pull requests. If a pull request passes/fails the test, travis automatically adds a corresponding annotation on the pull request. See here: http://about.travis-ci.org/blog/2012-09-04-pull-requests-just-got-even-more-awesome/

Anyway, I don't care so much about the method, as long as some testing happens. :-). I will take a look at doing this with tox and/or travis, and if I manage to get something useful going, I can just submit it as a PR as usual :-)

dusty-phillips commented 11 years ago

Indeed, some CI would be wonderful here. I believe tox was made with Travis (or maybe it was Jenkins) in mind, so it should be easy to integrate.

fingolfin commented 11 years ago

I tried running tests on Travis, but run into some problems due to a weird exception that in turn triggered a bug in py.test (or rather, in py). I was about to report this, but discovered that they already fixed it 2 days ago -- thanks to a report by Jed ;), see https://bitbucket.org/hpk42/py/issue/27 -- so I guess Jed has also be looking into this.

Anyway, I'll see if I can figure out the root cause for this exception.

fingolfin commented 11 years ago

See also https://travis-ci.org/fingolfin/gitifyhg/jobs/4395239 https://github.com/fingolfin/gitifyhg/blob/travis/.travis.yml

fingolfin commented 11 years ago

I figured out the cause for the problem (I had forgotten to install gitifyhg before running its tests -- d'oh, stupid). Now the tests run (but fail) for python 2.7 and currently four mercurial versions (the latest, and those used by most recent Ubuntu / Debian releases). It would be trivial to extend this also python2.6, and more mercurial versions. But first the test failures should be addressed.

They all come from checks like this:

  assert sh.git.push(_ok_code=1).stderr.find("master -> master (non-fast-forward)") > 0

For an example, see https://travis-ci.org/fingolfin/gitifyhg/jobs/4450360

I am not completely sure what causes this, but I guess it is related to how multiple apps now what to capture input and output: gitifyhg itself (input/output from mercurial), py.test, and finally the travis runner... but I am just guessing, so this might be nonsense.

The test code does make me wonder, though: If the message output changes, the test will fail. And a trivial way to change the output is to change the locale. Indeed, setting LANG=de_DE.UTF-8 changes the git pull error message... Interestingly, the suite is not affected; I can only guess that py.test performs some cleanup in the environment or indeed sets up a clean environment from scratch before running tests.

Still, perhaps using "--porcelain" for those git commands whose output we want to parse might be a good idea?

fingolfin commented 11 years ago

I added hg 1.6.4 and 1.8.4 to the travis.yml file. For some reasons I have not yet discovered, this causes all tests to fail. The main reasons seems to be that "bookmarks" was not an internal module back then, but rather an extension, and we unconditionally do

from mercurial.bookmarks import listbookmarks, readcurrent, pushbookmark

Moreover readcurrent does not even exist in 1.6.4 and 1.8.4 (or rather, it is there as bookmark_repo._bookmarkcurrent(), defined and instantiated in its reposetup() function...). We could try to work around that all, and attempt to support 1.6.x and 1.8.x. Or, we just say 1.9.x is the minimal supported version, and add an according test that refuses to install or start gitifyhg with older hg versions.

What should we do? Option 2 is of course much less work. But some people might be stuck with older hg versions...

dusty-phillips commented 11 years ago

Thanks for your work on this. I'm definitely of two minds on the question of what to support. I'd kind of rather the time spent supporting older versions be dedicated to other features, but I am also eager to make it as accessible as possible to all users.

I don't think anyone would be completely locked out of using gitifyhg if it requires newer versions of hg. They can always install gitifyhg into a virtualenv with easy_install -- it explicitly depends on mercurial and would pull in the latest version.

No matter what decision is made, setup.py should be updated to depend on the most recent version of mercurial that gitifyhg supports, whatever that might be. Mercurial>=2.4.2 might be a start, and then we can move it backwards as we test/fix/confirm that it runs on older versions.

fingolfin commented 11 years ago

According to the tests we run on Travis, it seems we support at least Mercurial 1.9.1. Of course we did not tests every version since then, but it seems unlikely to me that some versions inbetween causes extra problems. Of course there might still be problem hidden there -- but we will automatically discover those as we add more tests. And if any users run into problems with older hg versions, I would hope that they'd file a bug report, so that we can add additional tests cases (and possibly code fixes) for the code paths causing issues on older hg versions.

Anyway, for now it seemed like an improvement to change setup.py to require Mercurial >= 1.9.1 (better than not requiring any version). If we decide to implement compat with older versions later on, we can easily change this.

One thing that I still would like to do is run tests against more git versions. I have an experimental setup for that, the main blocker is that not enough git versions are readily available for Ubuntu 12.04 from PPAs :-). I thought about setting up my own with older versions... but for now, it would already be good to test against those git versions already available. I'll work on that.

fingolfin commented 11 years ago

I had a closer look at Mercurial 1.8.4 after I realized that it actually contains bookmarks, and not just as an extension. After working around some minor API changes, I discovered that a major stumbling block is that it does not seem to transfer bookmarks when cloning! IMHO that is a very good reason to require at least Mercurial 1.9.x.

So, if nobody objects, I think this issue can now be closed -- we currently say Mercurial >= 1.9.1 is required (which we could change to 1.9, I guess), and supporting older versions seems difficult. And we frequently test against multiple hg versions, too...

fingolfin commented 11 years ago

And for reference, if anybody wants to work on 1.8.x support, I committed my initial changes to make cloning / pulling work to https://github.com/fingolfin/gitifyhg/tree/hg-1.8-support

fingolfin commented 11 years ago

Since we have those travis tests up and running, and thus our test suite gets run all the time, I believe we can close this. I'll make sure to add new hg versions as they are released, and if possible, I'll also look into enhancing the tests to cover multiple git versions, too. But for now, we are covered.