Managing dependencies in Nox sessions with Poetry

[GuyHoozdis]

See the branch attached to this issue. Also see my notes on Hypermodern Python in Evernote. The section with the same name as this issue covers everything.

Come back and sort through those issues to successfully implement the last two sections of Chapter 3. I skipped the section on pre-commit. It might preclude this section, but still I want to explore and understand the ideas presented.

I haven't tried this yet, but just to make sure there wasn't something wonky with my source code you could try the author's source code for this chapter (or view changes) and check if you get the same errors.

[GuyHoozdis]

try the author's source code for this chapter (or view changes) and check if you get the same errors.

Yes, I do get the same errors.

.. but first, I had to poetry lock to update the lock file because I was getting an error when running poetry install

Note: This error originates from the build backend, and is likely not a problem with poetry but with typed-ast (1.4.1) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "typed-ast (==1.4.1)"'.

This package was a dependency of black. After updating the lock file, which moved to the most recent version of black just like my environment was using, I started getting the same error.

$ nox -s black
nox > Running session black
nox > Creating virtual environment (virtualenv) using python3.8 in .nox/black
nox > poetry export --dev --format=requirements.txt --output=/var/folders/9n/ftwzvy4j5d371p9r6715c4500000gn/T/tmp09kje6hk
The `--dev` option is deprecated, use the `--with dev` notation instead.
nox > python -m pip install --constraint=/var/folders/9n/ftwzvy4j5d371p9r6715c4500000gn/T/tmp09kje6hk black
nox > Command python -m pip install --constraint=/var/folders/9n/ftwzvy4j5d371p9r6715c4500000gn/T/tmp09kje6hk black failed with exit code 1:
DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement. Discussion can be found at https://github.com/pypa/pip/issues/8210
ERROR: Constraints cannot have extras

[notice] A new release of pip is available: 24.0 -> 24.1.1
[notice] To update, run: pip install --upgrade pip
nox > Session black failed.

So, it seems that it is a feature of pip that is causing the issue. I'm was using 24.0 when I got the error above.

---

Try matching the environment he uses in the article exactly. Python versions 3.8.2 and 3.7.7, then see this error persists. Check the article to see if he specifies the version of poetry used and if he does, then match that too.

[GuyHoozdis]

See the "Matching the Article's Python Versions Too" section of the same note for more details.

I spent a few hours trying, but couldn't get those versions of Python installed with pyenv. The following articles describe similar errors to what I was seeing. The solutions proposed did not work for me.

• https://stackoverflow.com/questions/50036091/pyenv-zlib-error-on-macos
• https://gist.github.com/aputs/1e898436c70c23ea739899a7942a720a
• https://stackoverflow.com/questions/50036091/pyenv-zlib-error-on-macos/54955286#54955286
• https://github.com/pyenv/pyenv/issues/1643

Not even for the python versions that they were using.

I was able to get 3.8.4 installed, pretty close to 3.8.2, which comes with pip 20.1.1 and so I gave the author's source another shot. I couldn't even get there though because I couldn't even get poetry installed. First I tried to use the same method he does - fetching get-poetry.py and piping that into python. No content was found at the URL. Then I tried pip install poetry. It selected version 1.8.3 and eventually threw an error.

Collecting filelock<4,>=3.12.2
  Downloading filelock-3.15.4-py3-none-any.whl (16 kB)
Collecting rapidfuzz<4.0.0,>=3.0.0
  Downloading rapidfuzz-3.9.3.tar.gz (1.6 MB)
     |████████████████████████████████| 1.6 MB 26.9 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/sully/.pyenv/versions/3.8.4/bin/python3.8 /Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/9n/ftwzvy4j5d371p9r6715c4500000gn/T/pip-build-env-u1vopawr/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- cmake
       cwd: None
  Complete output (53 lines):
  Collecting cmake
    Downloading cmake-3.29.6.tar.gz (30 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Installing backend dependencies: started
    Installing backend dependencies: finished with status 'error'
    ERROR: Command errored out with exit status 2:
     command: /Users/sully/.pyenv/versions/3.8.4/bin/python3.8 /Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/9n/ftwzvy4j5d371p9r6715c4500000gn/T/pip-build-env-napqnnce/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'ninja>=1.5' cmake
         cwd: None
    Complete output (37 lines):
    Collecting ninja>=1.5
      Downloading ninja-1.11.1.1.tar.gz (132 kB)
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
        Preparing wheel metadata: started
        Preparing wheel metadata: finished with status 'done'
    Collecting cmake
      Using cached cmake-3.29.6.tar.gz (30 kB)
    ERROR: Exception:
    Traceback (most recent call last):
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
        status = self.run(options, args)
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
        return func(self, options, args)
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 332, in run
        requirement_set = resolver.resolve(
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
        discovered_reqs.extend(self._resolve_one(requirement_set, req))
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
        abstract_dist = self._get_abstract_dist_for(req_to_install)
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 314, in _get_abstract_dist_for
        abstract_dist = self.preparer.prepare_linked_requirement(req)
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 487, in prepare_linked_requirement
        abstract_dist = _get_prepared_distribution(
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 90, in _get_prepared_distribution
        with req_tracker.track(req):
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/contextlib.py", line 113, in __enter__
        return next(self.gen)
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 149, in track
        self.add(req)
      File "/Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 116, in add
        raise LookupError(message)
    LookupError: https://files.pythonhosted.org/packages/16/fb/64d20d1a11552f708eaba491a7574631583355ff785ceb89138434ef4fe6/cmake-3.29.6.tar.gz#sha256=6a0c9574c4be92044fa9c373a1bddca4c960f37796a8727949bbc98fcada96a5 (from https://pypi.org/simple/cmake/) (requires-python:>=3.7) is already being built: cmake from https://files.pythonhosted.org/packages/16/fb/64d20d1a11552f708eaba491a7574631583355ff785ceb89138434ef4fe6/cmake-3.29.6.tar.gz#sha256=6a0c9574c4be92044fa9c373a1bddca4c960f37796a8727949bbc98fcada96a5
    WARNING: You are using pip version 20.1.1; however, version 24.1.1 is available.
    You should consider upgrading via the '/Users/sully/.pyenv/versions/3.8.4/bin/python3.8 -m pip install --upgrade pip' command.
    ----------------------------------------
  ERROR: Command errored out with exit status 2: /Users/sully/.pyenv/versions/3.8.4/bin/python3.8 /Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/9n/ftwzvy4j5d371p9r6715c4500000gn/T/pip-build-env-napqnnce/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'ninja>=1.5' cmake Check the logs for full command output.
  WARNING: You are using pip version 20.1.1; however, version 24.1.1 is available.
  You should consider upgrading via the '/Users/sully/.pyenv/versions/3.8.4/bin/python3.8 -m pip install --upgrade pip' command.
  ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/sully/.pyenv/versions/3.8.4/bin/python3.8 /Users/sully/.pyenv/versions/3.8.4/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/9n/ftwzvy4j5d371p9r6715c4500000gn/T/pip-build-env-u1vopawr/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- cmake Check the logs for full command output.
WARNING: You are using pip version 20.1.1; however, version 24.1.1 is available.
You should consider upgrading via the '/Users/sully/.pyenv/versions/3.8.4/bin/python3.8 -m pip install --upgrade pip' command.

(╯°□°)╯︵ ┻━┻

[GuyHoozdis]

I spent 2 hours trying to get 3.8.2 and 3.7.7 installed. No joy. At this point I think it would be better to update the technique so that it works with the python versions that do install.

Also, 3.7 is past End of Life support and 3.8 is towards the end of Security support.

[GuyHoozdis]

I did this for the documentation dependencies. It seems to work well. Why didn't the Hypermodern Python Project author do it that way instead of making install_with_constraings?