search

aio-libs / aiohttp

Asynchronous HTTP client/server framework for asyncio and Python
https://docs.aiohttp.org
Other
14.95k stars 1.99k forks source link

Refactor Cython usage #3693

Closed asvetlov closed 5 years ago

asvetlov commented 5 years ago

pip 19+ uses isolated virtual environments to install a package. It breaks a build, please read #3581 report

The idea is:

  1. We don't need Cython to be installed in venv if all .pyx files are already precompiled into .c sources.
  2. .c are already present in distribution tarball. Binary wheels don't need to compile sources at all.
  3. Precompilation step can be done in Makefile before installing aiohttp library.
  4. Pure-python installation should require AIOHTTP_NO_EXTENSIONS=1 env var. Otherwise, a compilation error is raised. It prevents a silent performance degradation if aiohttp installed on Alpine or OS without C compiler toolchain.

3694 addresses points 1-3, it can be backported to aiohttp 3.5.

The 4th bullet is not backward compatible. It should be implemented in a separate PR and land on aiohttp 4.0 only.

webknjaz commented 5 years ago

See also: https://github.com/pypa/pip/issues/6144

I think I've actually got it figured out in #3581. https://www.python.org/dev/peps/pep-0517/#in-tree-build-backends would work for us but it's not yet implemented by Pip.

The current implementation relies on an undefined requires filtering behavior as per https://github.com/pypa/pip/issues/6410.

So the only thing left to do is to put it into a separate dist. (And it works!)

asvetlov commented 5 years ago

manylinux is broken still but #3694 solves almost everything

webknjaz commented 5 years ago

I'd keep this open anyway. I'm still thinking of improvements.

asvetlov commented 5 years ago

Up to you, the PR will hang for years then. I have a feeling that supporting both brand new and 2-years-old pip versions is a good idea.

asvetlov commented 5 years ago

The issue is solved by adding explicit make cythonize step.