"Dist currently only works with Git or Mercurial repos" in a Mercurial monorepo

[paugier]

I try to produce a sdist of a package not in the root directory of a Mercurial repository. I get "Dist currently only works with Git or Mercurial repos" while this is in a Mercurial repository.

To reproduce:

hg clone https://foss.heptapod.net/fluiddyn/fluidfft
cd fluidfft/plugins/fluidfft-fftw
python3 -m pip install build
python3 -m build -s

I checked and it works fine with the corresponding Git repo (https://github.com/fluiddyn/fluidfft).

Is there a workaround?

[eli-schwartz]

The implementation for git knows how to detect being in a git repo and making a dist out of a subdirectory. It relies on git archive's support for making an archive of a subdirectory.

The implementation for hg only knows how to make a dist out of the entire repository. If it's possible to tell hg archive to only do a subdirectory that would be great -- patches welcome.

(Sadly I don't know much about mercurial. I tried to learn it several times but it never clicked.)

[paugier]

This happens here: https://github.com/mesonbuild/meson/blob/master/mesonbuild/mdist.py

The Mercurial support is indeed quite minimalist compared to what is done for Git. At least, there is something and it should not be very difficult to improve.

I also see (https://github.com/mesonbuild/meson/blob/master/mesonbuild/mdist.py#L225)

mlog.warning('dist scripts are not supported in Mercurial projects')

so dist scripts are actually not supported for Mercurial repos (see https://github.com/mesonbuild/meson/issues/12796#issuecomment-1917456514).

[paugier]

@eli-schwartz I'd like to try to improve the situation. Most of the code in GitDist is actually not specific to Git and should go into Dist. It would be great if I can simply test that I break nothing. I didn't find how to run the tests involving mdist.py. How can I do that?

[paugier]

I don't understand why there is this check: https://github.com/mesonbuild/meson/blob/master/mesonbuild/mdist.py#L201

            if is_git(sub_src_root):
                self.process_git_project(sub_src_root, sub_distdir)
            else:
                shutil.copytree(sub_src_root, sub_distdir)

It seems to be for subprojects for which the main meson.build file (which is in the main repo) is not versioned? I don't understand in which situation it can be useful and why it is even allowed (since creating a sdist from unversioned source is forbidden).

Is the line with copytree covered in a test?

[eli-schwartz]

It seems to be for subprojects for which the main meson.build file (which is in the main repo) is not versioned? I don't understand in which situation it can be useful and why it is even allowed (since creating a sdist from unversioned source is forbidden).

To be more specific: it is there for subprojects which are acquired via a [wrap-file], for which we assume that it is never modified by users under any circumstances ever -- we have a number of commands that will actually "blow away" the entire contents of such, re-initializing it from the checksummed tarball.

I do agree with you that that is kind of wishy-washy. It would probably be a good idea to actually enforce this by doing just that and extracting the wraps again from their canonical sources. It is more practical to do that now that we respect $MESON_PACKAGE_CACHE_DIR as we can use that internal machinery and just run meson subprojects download inside the dist root.

@eli-schwartz I'd like to try to improve the situation. Most of the code in GitDist is actually not specific to Git and should go into Dist. It would be great if I can simply test that I break nothing. I didn't find how to run the tests involving mdist.py. How can I do that?

I reformatted that all into classes in a146ee6946b4d5cde8b4cd6460260aedc5b438d2 in an effort to tame spaghetti code, but did not try too hard to make it as generic as it can.

One benefit is that now that it's factored out into classes, people can look at it and try to share enough logic to add git-specific features to hg as well. In fact, that was secretly my hope when I initially refactored it -- I just lacked the personal experience to do it myself.

In short, if you can refactor this and pull out git functions into generic shared ones, fantastic!

As far as testing goes, we have unittests/allplatformstests.py which can be run with ./run_unittests.py (if pytest is installed, it passes arguments as -k lists to pytest, so use ./run_unittests.py test_dist to run all dist-related tests).

You'll want to update test_dist_hg() to test what you implement, as well.