Move development-only dependencies in setup.py from `install_requires` to `extras_require[dev]`

YannickJadoul commented 1 year ago

Closes #1483.

I've installed abjad in a fresh, empty venv, and importing works. Then manually installing pytest makes all tests run. So I assume I have split up the dependencies correctly.

I've also looked for install -e and --editable, so I hope I have found all the places in the documentation that need to have [dev] added, but please do double check, as I do not know the library well.

YannickJadoul commented 1 year ago

On a separate note, I noticed that python_requires isn't specified in setup(...), so the following message does not work: https://github.com/Abjad/abjad/blob/3f62fc1a376060451b90ddb09763f8c223e23f94/setup.py#L20-L28

I have tested with Python 3.8, and in an old version, pip install abjad fails, rather than installing an old version.

Are you aware of this, or should I open another issue for this?

trevorbaca commented 1 year ago

The separation of dependencies looks good, and I've merged into main.

Re: the second point, what do you suggest? That error message in setup.py isn't doing much; maybe we should get rid of it?

YannickJadoul commented 1 year ago

Thanks, great! :-)

Re: the second point, what do you suggest? That error message in setup.py isn't doing much; maybe we should get rid of it?

@trevorbaca, well, the check does stop the library from being installed on Python < 3.10. But somewhere, you seem to have lost the python_requires flag?

So I would say you definitely need to add the python_requires again, to get it correct for future versions.

But because it's not there in some versions on PyPI, users won't automatically install the latest version that still works for them :-/ I can see two ways of going forward:

You assume that everyone now has support for python_requires (setuptools 24.2.0 and pip 9.0.0 are 6 years old, older than the release date of the oldest still-supported Python version 3.7). If you think that's good enough, you can remove the whole check and just rely on python_requires. This won't solve Python 3.9 users trying pip install abjad, though.
Or you update the error message and explicitly list "For Python 3.9, python -m pip install abjad<=x.y", such that it will work. (I used pyenv to easily install an old Python version and try out what happens, btw)

trevorbaca commented 1 year ago

Ok, great thanks for such a helpful answer: I'll add python_requires back, and then also update the check / error explicitly.

The code in main now works under Python 3.11, so this is the right time to make the changes before building Abjad 3.14, which I'll do tomorrow.

YannickJadoul commented 1 year ago

Thanks! Great to hear :-) So the current version will only work with 3.11, and not with 3.10 anymore? Then yes, then the best way is probably to make sure people know what to install for older versions of Python :-)

trevorbaca commented 1 year ago

Abjad 3.13 (the current version) only works with Python 3.10. But Abjad 3.14 will work with both Python 3.10 and Python 3.11; the transition to Python 3.11 was easy, and Abjad 3.14 doesn't (yet) use any Python 3.11-only features.

trevorbaca commented 1 year ago

Ok @YannickJadoul can you take a look at setup.py (in main) and maybe even install into a venv again?

Everything should be set to allow installation under any Python 3.10.x or 3.11.x (and refusing installation under other versions).

YannickJadoul commented 1 year ago

Alright, I've tried this out on 3.8, 3.9, 3.10, and 3.11, and things are +- as expected, I think:

3.8 and 3.9 fail to install from the local checkout, but also from PyPI:

Python 3.8.14

``` yannick@Delphi:~/abjad$ . venv-3.8/bin/activate (venv-3.8) yannick@Delphi:~/abjad$ python --version Python 3.8.14 (venv-3.8) yannick@Delphi:~/abjad$ pip install -e . Obtaining file:///home/yannick/abjad Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [3 lines of output] This is Abjad 3.14, which supports Python 3.10, 3.11. But it looks like you're trying to install Abjad with Python 3.8. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. WARNING: You are using pip version 22.0.4; however, version 22.3 is available. You should consider upgrading via the '/home/yannick/abjad/venv-3.8/bin/python -m pip install --upgrade pip' command. (venv-3.8) yannick@Delphi:~/abjad$ pip install abjad Collecting abjad Using cached abjad-3.13.tar.gz (419 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [17 lines of output] ========================== Unsupported Python version ========================== This version of Abjad requires Python 3.10, but you're trying to install it on Python 3.8. This may be because you are using a version of pip that doesn't understand the python_requires classifier. Make sure you have pip >= 9.0 and setuptools >= 24.2, then try again: $ python -m pip install --upgrade pip setuptools $ python -m pip install abjad This will install the latest version of Abjad which works on your version of Python. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. WARNING: You are using pip version 22.0.4; however, version 22.3 is available. You should consider upgrading via the '/home/yannick/abjad/venv-3.8/bin/python -m pip install --upgrade pip' command. ```

Python 3.9.14

``` yannick@Delphi:~/abjad$ . venv-3.9/bin/activate (venv-3.9) yannick@Delphi:~/abjad$ python --version Python 3.9.14 (venv-3.9) yannick@Delphi:~/abjad$ pip install -e . Obtaining file:///home/yannick/abjad Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [3 lines of output] This is Abjad 3.14, which supports Python 3.10, 3.11. But it looks like you're trying to install Abjad with Python 3.9. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. WARNING: You are using pip version 22.0.4; however, version 22.3 is available. You should consider upgrading via the '/home/yannick/abjad/venv-3.9/bin/python -m pip install --upgrade pip' command. (venv-3.9) yannick@Delphi:~/abjad$ pip install abjad Collecting abjad Using cached abjad-3.13.tar.gz (419 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [17 lines of output] ========================== Unsupported Python version ========================== This version of Abjad requires Python 3.10, but you're trying to install it on Python 3.9. This may be because you are using a version of pip that doesn't understand the python_requires classifier. Make sure you have pip >= 9.0 and setuptools >= 24.2, then try again: $ python -m pip install --upgrade pip setuptools $ python -m pip install abjad This will install the latest version of Abjad which works on your version of Python. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. WARNING: You are using pip version 22.0.4; however, version 22.3 is available. You should consider upgrading via the '/home/yannick/abjad/venv-3.9/bin/python -m pip install --upgrade pip' command. ```

3.10 and 3.11 nicely install, both from the local directory, as well as the older version from PyPI.

YannickJadoul commented 1 year ago

The one thing I could not check is what happens if you try to install with pip from PyPI for the new version. My guess is that, because python_requires is set, it will automatically skip Abjad 3.14 and actually download Abjad 3.13 and give that error message. So this means that the only case in which the new error message will show is when the version of pip is not high enough to understand the python_requires tag. So adding a line there to make sure people update pip might help?

Another thing: is there a reason you're not allowing Python 3.12 in python_requires="<3.12,>=3.10.*",? Python normally is backwards compatible for pure-Python libraries, so for your own future comfort, it might just be nice to have a minimum version but no maximum. This way, when a new Python version comes out, you don't need to urgently make a new Abjad release.

YannickJadoul commented 1 year ago

The one thing I could not check is what happens if you try to install with pip from PyPI for the new version.

Ah, no, wait. It's already on PyPI! OK, so then my guess was right:

yannick@Delphi:~/abjad$ python -m venv venv-2
yannick@Delphi:~/abjad$ . venv-2/bin/activate
(venv-2) yannick@Delphi:~/abjad$ python --version
Python 3.9.14
(venv-2) yannick@Delphi:~/abjad$ pip --version
pip 22.0.4 from /home/yannick/abjad/venv-2/lib/python3.9/site-packages/pip (python 3.9)

(venv-2) yannick@Delphi:~/abjad$ pip install abjad --verbose
Using pip 22.0.4 from /home/yannick/abjad/venv-2/lib/python3.9/site-packages/pip (python 3.9)
  Link requires a different Python (3.9.14 not in: '<3.12,>=3.10.*'): https://files.pythonhosted.org/packages/44/51/66916b778b7b4e40f9f31172a32ff09a6cf32c1d3af96559bb6cbc34b280/abjad-3.14.tar.gz#sha256=b621e9a9aa78c10cc7decca6906b4b2958377a320cef53e402967fc52ee1654c (from https://pypi.org/simple/abjad/) (requires-python:<3.12,>=3.10.*)
Collecting abjad
  Using cached abjad-3.13.tar.gz (419 kB)
  Running command python setup.py egg_info

  ==========================
  Unsupported Python version
  ==========================

  This version of Abjad requires Python 3.10, but you're trying to
  install it on Python 3.9.

  This may be because you are using a version of pip that doesn't
  understand the python_requires classifier. Make sure you
  have pip >= 9.0 and setuptools >= 24.2, then try again:

      $ python -m pip install --upgrade pip setuptools
      $ python -m pip install abjad

  This will install the latest version of Abjad which works on your
  version of Python.
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/yannick/abjad/venv-2/bin/python -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize

  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)

  __file__ = %r
  sys.argv[0] = __file__

  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"

  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/tmp/pip-install-kns2_kh0/abjad_b113be7667b84fda8fdc3e79aa2df8c2/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /tmp/pip-pip-egg-info-8ecdqwrr
  cwd: /tmp/pip-install-kns2_kh0/abjad_b113be7667b84fda8fdc3e79aa2df8c2/
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
WARNING: You are using pip version 22.0.4; however, version 22.3 is available.
You should consider upgrading via the '/home/yannick/abjad/venv-2/bin/python -m pip install --upgrade pip' command.

(venv-2) yannick@Delphi:~/abjad$ pip install abjad==3.14 --verbose
Using pip 22.0.4 from /home/yannick/abjad/venv-2/lib/python3.9/site-packages/pip (python 3.9)
  Link requires a different Python (3.9.14 not in: '<3.12,>=3.10.*'): https://files.pythonhosted.org/packages/44/51/66916b778b7b4e40f9f31172a32ff09a6cf32c1d3af96559bb6cbc34b280/abjad-3.14.tar.gz#sha256=b621e9a9aa78c10cc7decca6906b4b2958377a320cef53e402967fc52ee1654c (from https://pypi.org/simple/abjad/) (requires-python:<3.12,>=3.10.*)
ERROR: Could not find a version that satisfies the requirement abjad==3.14 (from versions: 1.1.1, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, 2.10, 2.11, 2.12, 2.13, 2.14, 2.15, 2.16, 2.17, 2.18, 2.19, 2.20, 2.21, 3.0.0rc1, 3.0.0, 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10, 3.11, 3.12, 3.13)
ERROR: No matching distribution found for abjad==3.14
WARNING: You are using pip version 22.0.4; however, version 22.3 is available.
You should consider upgrading via the '/home/yannick/abjad/venv-2/bin/python -m pip install --upgrade pip' command.

trevorbaca commented 1 year ago

Yeah, the python_requires="<3.12,>=3.10.*" probably should have been just python_requires=">=3.10.*". What was driving that is that Python 3.11 broke third-party quicktions. So importing Abjad 3.13 broke under Python 3.11. The solution there was just to remove quicktions from Abjad altogether (which is indeed the case in Abjad 3.14). But I think I walked around for a couple of days thinking "well, maybe ">=3.10" is a lie, because I it's possible that (say) Python 3.12 might repeat the process of break Abjad (via a third-party dependency) yet again; maybe it's more honest to limit Abjad to just the versions of Python that Abjad is actually tested to run on."

But I think this was wrong. And, in fact, rmakers and nauert just say python_requirest=">=3.10", which is what Abjad 3.14 (and later) will revert to doing, too.

trevorbaca commented 1 year ago

And thanks so much for all the testing, really appreciate it!

YannickJadoul commented 1 year ago

What was driving that is that Python 3.11 broke third-party quicktions. So importing Abjad 3.13 broke under Python 3.11. The solution there was just to remove quicktions from Abjad altogether (which is indeed the case in Abjad 3.14).

Ah, right. But well, I guess quicktions is to blame, then, and things would work when they fix their "lie" (i.e., provide a version that works on 3.11). But yeah, I see your point and at any rate, version management is hard, so I don't really know for sure either.

And thanks so much for all the testing, really appreciate it!

My pleasure! Glad to help out a tiny bit. And thank you for attending to these issues! :-)

Abjad / abjad

Move development-only dependencies in setup.py from `install_requires` to `extras_require[dev]` #1484