Build without wheels (Linux)

[adigitoleo]

Hi, not really an issue but more of a question. I'm trying to install from source, partly to learn and partly because I work mostly on Linux at the moment. I would prefer to use setuptools and avoid pip/wheels if possible (to manage everything through pacman). I've read the docs and checked setup.py but I'm probably missing something. I have all of the dependencies:

python-pyqt5: 5.15.1-2
python-ply: 3.11-5
python-kiwisolver: 1.3.1-1
python-atom: 0.5.2-1

However, when I try to build with python setup.py build I get distutils errors, which I think are related to the C extensions:

WARNING: The wheel package is not available.
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-jjlg2tia/cppy/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-jjlg2tia/cppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-zs557i_a
       cwd: /tmp/pip-wheel-jjlg2tia/cppy/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for cppy
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/setuptools/installer.py", line 126, in fetch_build_egg
    subprocess.check_call(cmd)
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpz26ziv4g', '--quiet', 'cppy>=1.1.0']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "setup.py", line 154, in <module>
    setup(
  File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 152, in setup
    _install_setup_requires(attrs)
  File "/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
    dist.fetch_build_eggs(dist.setup_requires)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 673, in fetch_build_eggs
    resolved_dists = pkg_resources.working_set.resolve(
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 764, in resolve
    dist = best[req.key] = env.best_match(
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1049, in best_match
    return self.obtain(req, installer)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1061, in obtain
    return installer(requirement)
  File "/usr/lib/python3.8/site-packages/setuptools/dist.py", line 732, in fetch_build_egg
    return fetch_build_egg(self, req)
  File "/usr/lib/python3.8/site-packages/setuptools/installer.py", line 128, in fetch_build_egg
    raise DistutilsError(str(e)) from e
distutils.errors.DistutilsError: Command '['/usr/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpz26ziv4g', '--quiet', 'cppy>=1.1.0']' returned non-zero exit status 1.

The idea was to run python setup.py install --skip-build afterwards. I had a look at the setuptools docs briefly and maybe I need to run the sdist command somewhere first, but I couldn't really figure it all out. I can ask on linux forums as well but I was just wondering if there is anything obvious that I am missing, like do I need cppy installed?

[MatthieuDartiailh]

cppy is indeed needed (it is a build only dependency which is listed in setup_requires). Note that among the runtime dependency (which are listed in the docs) you are missing qtpy, and bytecode. Let me know if you have any additional question and feel free to submit a PR to update the docs (otherwise I will do it at one point).

[adigitoleo]

Thank you so much! I actually forgot to list qtpy yesterday. I'll try again soon with cppy and see how it goes.

[adigitoleo]

you are missing [...] bytecode

You mean this module right? I checked again and I don't think it's listed in the enaml docs. Once I get it working I might submit a PR as you suggest.

[adigitoleo]

Alright I got an arch package for python-bytecode working and fixed the dependencies, now I'm able to build enaml. However when I try to run the tests, pytest doesn't seem to be able to import enaml properly and fails with ModuleNotFoundError: No module named 'enaml'. I was just running pytest tests in the project root.

[adigitoleo]

Just another little update: I tried python -m pytest tests instead since that is how the tests are run in the ci.yml but I got a different import error: ModuleNotFoundError: No module named 'enaml.colorext'.

Edit: To clarify, I have installed the same testing modules as found in ci.yml:

python-pytest: 6.1.2-1
python-pytest-qt: 3.3.0-2
python-pytest-cov: 2.10.1-1

(although I probably don't need the coverage plugin if I'm not using the --cov flag)

[MatthieuDartiailh]

Thanks for the updates, and sorry the documentation is indeed lagging behind the state of setup.py (and you did pick the right bytecode).

How do you install enaml for testing ? python setup.py build does not install the package per se, you need python setup.py install for that. Other than that, the testing issue you are having is weird. I cannot reproduce locally in a clean environment with a clean repo (no compiled modules) and installing enaml from pypi. Can you manually check that your install contains the expected extension packages ? If you first packaged enaml in an arch package and then installed that make sure the extension package are properly handled.

[adigitoleo]

sorry the documentation is indeed lagging behind the state of setup.py (and you did pick the right bytecode)

No problem, I can see that this is a very active project so I'm not surprised. Thanks for the confirmation.

How do you install enaml for testing

You are right, I only ran python setup.py build before running the tests. However, after further reading I now understand that the actual complete package including C extensions is compiled to a new build/ directory beside the source tree. It was a matter of running the tests from the right location.

I will do my best to maintain an up-to-date build recipe on the arch user repo now (there was an old one but it hasn't been updated in 2 years). Thanks for the help.

[MatthieuDartiailh]

The project is not that active but I do my best to keep it up to date. Thanks for packaging for arch. If you have time for a PR to update the docs I would really appreciate it.

[adigitoleo]

Got a bit swamped now, PR would be next week at the earliest. I'll see what I can do.