Restore custom install of exe files

[ozagordi]

This was deleted by mistake in a previous PR. It is now possible to install shorah after building. Build first with meson build; cd build; ninja. Then, in the project root folder, pip install . will move the exe files and install shorah package. For development of python code one can use or pip install -e .

[DrYak]

I'm trying to test your branch, but I must be doing something wrong :

• if I follow your instructions ( meson build; pushd build; ninja; popd ; pip install .) only $(prefix)bin/shorash and $(prefix)lib/python3.5/site-packages/ShoRAH-1.1.4.dev3+gcb7c3c7-py3.5.egg-info get moved around.
• nothing appear in $(srcdir)src/shorah/bin/

I can't understand why I can't successfully use your meson setup.

(But otherwise, tests pass)

[ozagordi]

What about pip install -e ./?

[DrYak]

with the -e option :

• this time the executables DO get copied to $(srcdir)src/shorah/bin/
• but only $(prefix)bin/shorah amd $(prefix)lib/python3.6/site-packages/ShoRAH.egg-link happens in the installation target directory

(Which I think should make sense for "Install a project in editable mode from a local project path", stuff is supposed to remain in the project directory ?)

[DrYak]

(Also indirectly related to the setuptools reworking : now shorah insist on looking for binaries in a bin subdir of its module, but autotools are still installing them in $(prefix)/bin, not $(PYTHONPATH)/shorah/bin)

[ozagordi]

With -e the code is supposed to remain in the project directory, yes. This is convenient in development: any change in the code can immediately be tested, no need to install after every edit.

I don't really understand why $(srcdir)/src/shorah/bin is not populated by pip install ., the code seems correct to me. I will give another look.

Where to put binaries is crucial here! My python code looks into $(srcdir)/src/shorah/bin/ only. If we keep autotools, they end up somewhere else. I will think of an alternative from the Python side.

[DrYak]

A viable alternative in python would be to fallback to normal $PATH searches (or use $PATH first). So that it also works with packages from conda, RPMs, etc.

Or an autotool alternative would be to them creat a symblink in $(PYTHONPATH)/shorah/bin pointing to $(srcdir)/src/shorah/bin (so literally ../../../../bin/ to make it position independent).

But otherwise this PR doesn't break anything existing, so it can be merged as soon as you're happy with it.

[ozagordi]

Just opened #51. Looking in src/shorah/bin first, and in PATH if executables are not there. I only tested it with pip install -e ./, not with autotools.

[DrYak]

I think the problem you had in the other thread might just be related to this "c++ bin file not getting copied as expected" issue.

[DrYak]

With the latest e0df210 merge, in autotool-mode everything works as it should (installed in a fresh new prefix, amplicon test example passes), without any manual linking needed.

[DrYak]

With the latest e0df210 merge, in meson + pip, production build, it still craps out with :

Traceback (most recent call last):
  File "/home/dryak/project/shorah/test-meson/bin/shorah", line 11, in <module>
    load_entry_point('ShoRAH==1.1.4.dev7+ge0df210', 'console_scripts', 'shorah')()
  File "/home/dryak/miniconda3/envs/shorah/lib/python3.6/site-packages/pkg_resources/__init__.py", line 572, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/home/dryak/miniconda3/envs/shorah/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2755, in load_entry_point
    return ep.load()
  File "/home/dryak/miniconda3/envs/shorah/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2408, in load
    return self.resolve()
  File "/home/dryak/miniconda3/envs/shorah/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2414, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'shorah'

[DrYak]

With the latest e0df210 merge, in meson + pip, debug build, and adding a ninja install step :

    meson --prefix=${PREFIX} build ;
    pushd build ;
    ninja 
    ninja install
    popd
    PYTHONPATH=${PREFIX}/lib/python3.6/site-packages/ pip install --prefix ${PREFIX} -e .

everything works similarly to auto-tools (with shorah picking up the c++ bins from ${PREFIX}/bin), (amplicon test example passes again, in yet another fresh prefix).

(I mean, passes while displaying giant walls of debugging information, as expected from a debug build)

[DrYak]

So basically, the branch is in a mergeable state without breaking much things (Autotool work as expected, meson works for editable/debug installs).

But somewhat meson/pip production builds are still breaking.

[ozagordi]

Let’s merge this and declutter a bit the PRs. I will look into meson + pip in another branch.