Open vancromy opened 1 year ago
The discussion over at #3214 may be helpful.
I've since moved my projects to a pure-pyproject.toml
approach in order to stay on the bleeding edge of project packaging, but there is massive drawback: building packages in editable mode is an awful, awful experience. Currently, with pure pyproject.toml
, the only way to allow editable mode is to use pip install -e . --config-settings editable_mode=strict
, which typically creates a new folder of symlinked files that it treats as the actual package source. The issue for me (which is probably unrelated to setuptools) is that these symlinks constantly break in Windows with no warning, which means a dangerous work environment where I'm not sure which files have which edits.
I'd love to hear some more experienced developers chime in though. Is pure-pyproject.toml
still in its infancy and not recommended?
@mcrumiller I am curios in which sense the is a awful, awful experience
I have been using it for some time. The one major issue that I am aware of is https://github.com/pypa/setuptools/issues/3518 which breaks IDE / static checker integration.
The recommended way to work around this issue for now seems to be to use the legacy behaviour
The new editable install path however, depends on the implementation of pep660 and not pep621 as far as I am aware. You will get pep660 behaviour if you define a build back end in pyproject.toml but should be independent of whether you put the rest of your config in setup.py/setup.cfg or pyporject.toml
E.g. as long as you have
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
or similar in your pyproject.toml file
@jenshnielsen I was a little gushy in my over-use of the word awful, I apologize. Here are my main gripes:
setup.py
(or I haven't figured out how to use it otherwise)build/__editable__*
directories.build/__editable__*
directories, but 2) you can't set breakpoints in the same files you're working on, making the process pretty clunky.__init__.py
now requires a reinstall of the package (not a big issue).I admit I am quite the noob on build backends so I'm not here to complain but rather see if others have gone through and resolved any of my issues outlined above.
Thanks, those sounds like they are related to #3518 You should be able to get the legacy behavior with
pip install -e . --config-settings editable_mode=compat
As far as I understand that does not depend on the existence of the setup.py file
@jenshnielsen I swear I tried compat mode in the past and pylance failed to recognize editable installs--but now it works! Thank you so much.
Spoke too soon: compat mode doesn't seem to work at all:
(.venv_brr) C:\Projects>pip install -e ./python-toolbox --config-settings editable_mode=compat
<installation output>
Successfully installed python-toolbox-1.1.5
(.venv_brr) C:\Projects>python -c "import toolbox"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'toolbox'
I tried using SETUPTOOLS_ENABLE_FEATURES="legacy-editable"
as suggested here but I get the following error:
ERROR: Project file:///C:/Projects/python-toolbox has a 'pyproject.toml' and its build backend is missing the
'build_editable' hook. Since it does not have a 'setup.py' nor a 'setup.cfg', it cannot be installed in editable
mode. Consider using a build backend that supports PEP 660.
Any suggestions?
I assume that python-toolbox is https://github.com/cool-RR/python_toolbox . It seems like that is a pure setup.py project? To me it looks like compat mode is not working correctly in that case. Not sure it that is a bug but I would think so? (I am not a setuptool dev just a user)
Sorry nope, it's my own local package located in C:\Projects\python-toolbox
Ok could you share the pyproject.toml file from that project?
ERROR: Project file:///C:/Projects/python-toolbox has a 'pyproject.toml' and its build backend is missing the 'build_editable' hook. Since it does not have a 'setup.py' nor a 'setup.cfg', it cannot be installed in editable mode. Consider using a build backend that supports PEP 660.
Any suggestions?
Update the version of pip
/setuptools
you are using/have installed?
Here's the gist of it:
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
authors = [
{ name = "me", email = "my.email@com.com" },
]
name = "python-toolbox"
description = "Python toolbox"
dynamic = ["version"]
dependencies = [
"my_dependencies"
]
[project.scripts]
script_name = "module:function"
[tool.setuptools]
include-package-data = true
packages = [
"toolbox",
"toolbox.subpackages",
"my_subpackages"
]
[tool.setuptools.package-dir]
toolbox = "src/python/toolbox"
[tool.setuptools.package-data]
"toolbox" = "my_resource"
"toolbox.subpackage" = "subpackage_resource"
[tool.setuptools.dynamic]
version = { attr = "toolbox.__version__" }
readme = { file = ["README.md"] }
My project is setup like the following:
python-toolbox
|--src
|--python
|--toolbox
|--__init__.py
|--other files
@mcrumiller @jenshnielsen Happy to see you guys hash out some kinks with editable installs using pyproject.toml but that seems like it should perhaps be in a dedicated issue specifically for addressing editable install issues when using a pyproject.toml
file.
I'm afraid this discussion may lead to the focus getting shifted away from the core issue which was a call for more clarity and better guidance on which type of config file to go for when setting up or updating a Python package to meet today's packaging standards.
I do thank you for pointing me to #3214 because I learnt that setup.cfg
files may become obsolete. I think that just strengthens my case for requiring a clearer stance in the docs as to what developers should be doing.
Agreed, @mcrumiller perhaps it makes more sense for you to create a new bug for the issue that you are seeing with the information from here
Sorry to derail the conversation @vancromy. If anyone is interested I've opened #3767 for my own issue.
@vancromy, I am sorry to disappoint, but I don't think you are going to find objective technical reasons for using one or the other.
If I am not wrong the setup.cfg
format evolved first (even from outside setuptools
). The motivation why pyproject.toml
exists is kind of external to the setuptools
implementation itself, and more related to standardisation efforts.
Using pyproject.toml
, however, imposes constraints (specially around dynamic
) to the build. Some users are not comfortable with these constraints, so I don't think there should be a "black and white" recommendation.
If you are OK with these constraints, my personal recommendation (and I don't speak for setuptools here) is:
src
-layout that is good for autodiscovery and editable installation. That will make your life easierpyproject.toml
and its [project]
table to add metadata. If you are using a standard src
-layout, autodiscovery will do most of the work for you and there will be no need to configure [tool.setuptools]
.setuptools-scm
, so you are sure all the files under version control are included in the final distribution. This also helps with versioning - you can make the git tags the single source of truth for versions.setup.py
file (alongside pyproject.toml
, e.g.:
from setuptools import setup, Extension
setup(ext_modules=[Extension(...), ...])
@abravalheri is there any update on the beta status of [tools.setuptools] in pyproject.toml. I have a few projects that uses pyproject.toml but I have kept around setup.cfg for the setuptools specific config. This is mainly for stuff like zip_safe and package_data since I am not using setuptools-scm. It would be very nice to be able to remove the setup.cfg file all together for these projects
Any update?
@abravalheri This is exactly what I ended up doing except for part 4. I was able to get Cython working without a setup.py. It still requires a .py file, but it doesn't rely on setup.py being special. This essentially makes it so I have a pure pyproject.toml solution which is what I think the future is going to be. For anyone interested in an example you can checkout my repo https://github.com/MoseleyBioinformaticsLab/MESSES.
Summary
Hello,
As a newcomer to the python packaging world I was redirected from python.org to the setuptools documentation because python recommended it as the first choice for packaging. I had a quick read through the documentation and see that there are three different files for specifying project metadata/settings
setup.py
(for legacy projects - to be deprecated and kept to a bare minimum),setup.cfg
(as I understood from the docs, some sort of successor of setup.py but with no code execution) and finallypyproject.toml
. On theConfiguring setuptools using pyproject.toml files
page, the documentation mentions that since PEP 621 the python community has adopted the pyproject.toml as the go-to config file for specifying distribution/project metadata/settings. I therefore am left a bit confused as to what the use case is for thesetup.cfg
file and I did not feel like the documentation properly explained this. There is a separate page forConfiguring setuptools with setup.cfg files
but it doesn't address the overlap/difference between the two. I was therefore hoping that the experts in this community could clarify this and potentially have a separate page in the documentation (e.g.setup.cfg vs pyproject.toml
) describing:I had a quick scan and search of the GitHub issues to see if this had been brought up before but got no hits on my search so apologies if this is a dupe.
Note: I could have made this a discussion but specifically chose to opt for an issue because I felt an improvement to the docs was necessary for clarity's sake.
Looking forward to hearing from the experts! YVC
OS / Environment
N/A
Additional Information
For newbies like myself and potentially also intermediate/advanced developers, this clarity would hopefully ensure we are all going in the same direction in terms of the future of python packaging. :)
Code of Conduct