Cython builds dropped in 1.2.1

[gorlins]

Hi!

Thank you for this library, I've been using it for a while.

I noticed that the cython build is not triggered since 1.2.1. I'm using a clean pipenv environ in python 3.7 on a mac:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
cython = "*"

[packages]
geohash-hilbert = "==1.2.0"

[requires]
python_version = "3.7"

(hilbert) ➜  pip freeze
Cython==0.29.16
geohash-hilbert==1.2
(hilbert) ➜  python -c "from geohash_hilbert._hilbert import *; print(CYTHON_AVAILABLE)"
True

(hilbert) ➜  hilbert pipenv install geohash-hilbert==1.2.1
Installing geohash-hilbert==1.2.1…
Adding geohash-hilbert to Pipfile's [packages]…
✔ Installation Succeeded
Pipfile.lock (6bc29a) out of date, updating to (d6a1f1)…
Locking [dev-packages] dependencies…
✔ Success!
Locking [packages] dependencies…
✔ Success!
Updated Pipfile.lock (6bc29a)!
Installing dependencies from Pipfile.lock (6bc29a)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:00
(hilbert) ➜  hilbert python -c "from geohash_hilbert._hilbert import *; print(CYTHON_AVAILABLE)"
False

Still missing in 1.3.1

Installing geohash-hilbert==1.3.1…
Adding geohash-hilbert to Pipfile's [packages]…
✔ Installation Succeeded
Pipfile.lock (26ebc9) out of date, updating to (6bc29a)…
Locking [dev-packages] dependencies…
✔ Success!
Locking [packages] dependencies…
✔ Success!
Updated Pipfile.lock (26ebc9)!
Installing dependencies from Pipfile.lock (26ebc9)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:000:00
(hilbert) ➜  hilbert
(hilbert) ➜  hilbert python -c "from geohash_hilbert._hilbert import *; print(CYTHON_AVAILABLE)"
False

Switching back

(hilbert) ➜  hilbert pipenv install geohash-hilbert==1.2.0
Installing geohash-hilbert==1.2.0…
Adding geohash-hilbert to Pipfile's [packages]…
✔ Installation Succeeded
Pipfile.lock (d6a1f1) out of date, updating to (26ebc9)…
Locking [dev-packages] dependencies…
✔ Success!
Locking [packages] dependencies…
✔ Success!
Updated Pipfile.lock (d6a1f1)!
Installing dependencies from Pipfile.lock (d6a1f1)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:00
(hilbert) ➜  hilbert python -c "from geohash_hilbert._hilbert import *; print(CYTHON_AVAILABLE)"
True

[tammoippen]

Hi @gorlins,

Thank you for the kind words! In version 1.2.1 i changed the dev dependency manager from pipenv to poetry. All else should be the same. I guess, what you experienced is a missing cython during install (and maybe some caching?). The cythonize happens during execution of the setup.py script, i.e. cython has to be in the PATH at that time. With pipenv / poetry i do not know the install order of dev / prod dependencies and whether dev deps will be available for prod deps on install ( i guess not). To be sure dependencies will be cythonized, cython should be installed globally or in the corresponding venv before installing geohash-hilbert. I hope this helps.

Best, Tammo

[gorlins]

hmm... i think you are partially right, on further inspection fresh rebuilds of an environ even with Cython in a pipfile --dev and the cython deps do not work, even at 1.2.0. However, reinstalling that dependency fixes the issue. I thought --dev deps were supposed to be installed first via pipenv, but it appears this is not working in this case.

However, even with Cython already in the pipenv, I'm not able to install >=1.2.1 and get the cython extensions to work. I'm not fully a master of pip cache internals, but I think this shows that the build is not working properly:

➜  conda create -n testcy python=3.7 pip cython
➜ conda activate testcy
➜ pip freeze
certifi==2019.11.28
Cython==0.29.15
(testcy) ➜ pip install --no-cache-dir geohash-hilbert
Collecting geohash-hilbert
  Downloading geohash-hilbert-1.3.1.tar.gz (2.2 MB)
     |████████████████████████████████| 2.2 MB 1.7 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: geohash-hilbert
  Building wheel for geohash-hilbert (PEP 517) ... done
  Created wheel for geohash-hilbert: filename=geohash_hilbert-1.3.1-cp37-cp37m-macosx_10_9_x86_64.whl size=2218004 sha256=da99a1afc0fa8bf2344b478a0b282a16988e5cb0b6aee1cc201388877e169067
  Stored in directory: /private/var/folders/53/_y8h2sj95rl1v2y10t24dyhcmdbs20/T/pip-ephem-wheel-cache-0mowd50i/wheels/f3/14/48/bb531ad34216ee2c9573b66f8fd48e00d0cc603e556d18fc14
Successfully built geohash-hilbert
Installing collected packages: geohash-hilbert
Successfully installed geohash-hilbert-1.3.1
➜  python -c "from geohash_hilbert._hilbert import *; print(CYTHON_AVAILABLE)"
False
➜ pip install --no-cache-dir geohash-hilbert==1.2.0
Collecting geohash-hilbert==1.2.0
  Downloading geohash-hilbert-1.2.tar.gz (21 kB)
Building wheels for collected packages: geohash-hilbert
  Building wheel for geohash-hilbert (setup.py) ... done
  Created wheel for geohash-hilbert: filename=geohash_hilbert-1.2-cp37-cp37m-macosx_10_9_x86_64.whl size=26116 sha256=4cb83afc83e3f5563ea50cc8051df646b6c4cd72ef44abe7e8ad2f1fdfa9f0c6
  Stored in directory: /private/var/folders/53/_y8h2sj95rl1v2y10t24dyhcmdbs20/T/pip-ephem-wheel-cache-_rauaa8i/wheels/50/63/82/f39ce3c9c87dce9615355a4bd67f3d58e026172d3b93992e1e
Successfully built geohash-hilbert
Installing collected packages: geohash-hilbert
  Attempting uninstall: geohash-hilbert
    Found existing installation: geohash-hilbert 1.3.1
    Uninstalling geohash-hilbert-1.3.1:
      Successfully uninstalled geohash-hilbert-1.3.1
Successfully installed geohash-hilbert-1.2
(testcy) ➜  advanced_python git:(master) ✗ python -c "from geohash_hilbert._hilbert import *; print(CYTHON_AVAILABLE)"
True

[tammoippen]

mmmh.... after a lot of trail and error I found this: https://github.com/pypa/pip/issues/6144

Basically, since pip 19.0, they use isolated builds, if a pyproject.toml is present (since geohash-hilbert >=1.2.1) - hence the pip install does not see cython. For now, you can either install pip 18.1 or wait a bit until i create the next release with the pyproject.toml removed.

[tammoippen]

Can you please test, whether v1.4.0 works for you.

[gorlins]

Sorry I dropped out to prioritize other things, we can test soon I think. Thank you!