Upload wheels to pypi

[eltoder]

As far as I can tell, currently green only uploads sdist to pypi: https://pypi.org/simple/green/ This makes the installation slower, as everyone has to build their own wheels on demand. It will be much faster if green provides the wheels. Since green is pure python, this should be quite easy,

[sodul]

This twosigma link is broken for me. Is that a private cache at your company?

I'm not an expert with python packaging and releasing on pypi, my first time was when I added the automation to release directly from GH Actions a few months back. I assume adding wheels is not a lot of work, we just need to figure out the extra configuration required for this.

[sodul]

@eltoder before we plan this work. I took a look at this document that covers sdist vs wheels, and I'm not sure that it would be a big difference in speed in the case of green.

https://realpython.com/python-wheels/

From what I read, the reason wheels are faster is that C code is precompiled for the target systems, while sdist are pure source. With green all the code is pure python and since green embraces the KISS principle, it is a rather small package for which generating the pyc files should not be significant.

For example the most recent 4.0.2 release has a small 77kB source distribution file: https://pypi.org/project/green/#files

Compare that to pytest which has a source package at 1.4MB and a 440kB binary wheel ... this is definitely worth doing there.

https://pypi.org/project/pytest/#files

[eltoder]

Sorry, fixed the link.

It's a significant difference even for pure python packages. pip always installs wheels, so if you don't provide one, it has to build it from source. This always takes more time compared to having a wheel. You can time it yourself:

$ time ./venv/bin/pip install green-4.0.2-py2.py3-none-any.whl
Processing ./green-4.0.2-py2.py3-none-any.whl
Requirement already satisfied: coverage in ./venv/lib/python3.10/site-packages (from green==4.0.2) (7.5.1)
Requirement already satisfied: colorama in ./venv/lib/python3.10/site-packages (from green==4.0.2) (0.4.6)
Requirement already satisfied: unidecode in ./venv/lib/python3.10/site-packages (from green==4.0.2) (1.3.8)
Requirement already satisfied: setuptools in ./venv/lib/python3.10/site-packages (from green==4.0.2) (65.5.0)
Requirement already satisfied: lxml in ./venv/lib/python3.10/site-packages (from green==4.0.2) (5.2.1)
Installing collected packages: green
Successfully installed green-4.0.2

real    0m1.557s
user    0m1.456s
sys     0m0.097s

Compared to

$ time ./venv/bin/pip install green-4.0.2.tar.gz
Processing ./green-4.0.2.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: coverage in ./venv/lib/python3.10/site-packages (from green==4.0.2) (7.5.1)
Requirement already satisfied: setuptools in ./venv/lib/python3.10/site-packages (from green==4.0.2) (65.5.0)
Requirement already satisfied: lxml in ./venv/lib/python3.10/site-packages (from green==4.0.2) (5.2.1)
Requirement already satisfied: unidecode in ./venv/lib/python3.10/site-packages (from green==4.0.2) (1.3.8)
Requirement already satisfied: colorama in ./venv/lib/python3.10/site-packages (from green==4.0.2) (0.4.6)
Building wheels for collected packages: green
  Building wheel for green (pyproject.toml) ... done
  Created wheel for green: filename=green-4.0.2-py2.py3-none-any.whl size=78441 sha256=8ed239fe789836131dc4de70fa20f1d563849f89fc68a57d974760e56e3e045c
  Stored in directory: /home/eltoder/.cache/pip/wheels/d7/fe/47/bcbdee1bdef7dade09d80451280b6574d99255f2da1cac9e99
Successfully built green
Installing collected packages: green
Successfully installed green-4.0.2

real    0m8.931s
user    0m8.226s
sys     0m0.427s

So compared to a wheel, installing from an sdist takes extra ~7.5 seconds (6 times slower).

[sodul]

Thanks for benchmarking, shaving several seconds and not just a few milliseconds is worth it, especially when a bunch of other things will be installed.

We will have to build a new green-<version>-py3-none-any.whl file, I'll see if I can find other projects doing this while using pypa/gh-action-pypi-publish as examples.