Closed ksamuel closed 4 years ago
Fascinating reply from @sdispater in January to a qustion about getting poetry to use setuptools_scm "as a plugin or as direct integration" on twitter:
A plugin system is in the works and should be available in the 1.1 release. You can take a look at the WIP Pull Request here: #1237 No ETA yet. I would like to release the 1.1 version by the end of the quarter but it will be a big release so there is no guarantee that we will make it in time.
@madig I slightly disagree with your characterization as the information would exist twice, in pyproject.toml and another file therefore it is a duplication managed by poetry in my book, but since it's your idea you are more likely to understand it than I am, so I incorporated your description above.
@piccolbo you are correct, but the duplication isn't as bad as you don't usually need to care about it.
The pyproject.toml file must not be loaded into the project itself, because when the project is for production, the pyproject.toml file does not exist. You can send pyproject.toml with the project, but that would be very dirty.
The most appropriate way is to use a script to change the versions, in development mode, with that I created a small package to accomplish this feat. It's called EQVersion.
If anyone wants to take a look, visit: https://pypi.org/project/eqversion
@williamcanin ~you can distribute the pyproject.toml if you would like, as in the option is there to do it.~
EDIT: this is actually incorrect as per: #144 (comment)
@williamcanin you can distribute the pyproject.toml if you would like, as in the option is there to do it.
@adithyabsk how?
@williamcanin ~Add it to the include section of your pyproject.toml
~
EDIT: this is actually incorrect as per: https://github.com/python-poetry/poetry/issues/144#issuecomment-607321155
[tool.poetry]
...
include = [
"pyproject.toml"
]
@williamcanin Add it to the include section of your
pyproject.toml
[tool.poetry] ... include = [ "pyproject.toml" ]
@adithyabsk This is not guaranteed and safe. This would cause "pyproject.toml" to be sent to the "site-packages" folder. If another project has the same import configuration, it will replace the "pyproject.toml" of the other project.
@williamcanin just checked, and you're right
@adithyabsk Yes, sad, but it is true. So I created that little package to automate for me. With that I can create a Makefile for my production, for example:
:build
eqversion && poetry build
That was the temporary way to resolve this redundancy that I found.
My 2 cents:
I was quite used to add a __version__
in all my __init__.py
and use it as the "truth".
As I wanted to give poetry a go, I started to read quite a few discussions/issues/PR here and a bit elsewhere and stumbled upon #1036, there the solution given by @chrahunt seems indeed to be the way to go.
I tested quickly this with a single module and a package projects and it's working as intended (for >3.8.3 or backports of course):
from importlib import metadata
import toml
try:
__version__ = metadata.version(__package__)
except metadata.PackageNotFoundError:
__version__ = toml.load("pyproject.toml")["tool"]["poetry"]["version"] + "dev"
After testing this, I have to admit I see the sense in not having any more any metadata in the code and use this standard library import if needed (cli version or whatever).
Just wanted to comment as this read session helped me quite a bit :)
edit: added toml based info when in dev
Closing this as importlib.metadata
is the way to go. A possible pattern to support python <3.8 and >=3.8 I described here.
Closing this as importlib.metadata is the way to go. A possible pattern to support python <3.8 and >=3.8 I described here.
If this is the recommended solution, then poetry new
should be updated to use it. Currently, the poetry new
command adds __version__
to __init__.py
, but then poetry version
doesn't update it. This incompatibility between the two commands is confusing for me, and I imagine other newcomers.
Cross-posting my comment from: https://github.com/python-poetry/poetry/pull/2366#issuecomment-849615115 as it seems it's relevant here too.
Sorry if it feels like spam, but I think people subscribed to this issue might not be subscribed to that one and could miss this.
I just made a plugin (available with Poetry 1.2.0a1
and above) to read the version from an __init__.py
file:
# __init__.py
__version__ = "0.1.0"
or from a git tag, set with a GitHub release or with:
$ git tag 0.1.0
I thought this could be useful for others here too.
This plugin also has support for updating __version__
: https://github.com/mtkennerly/poetry-dynamic-versioning
Reading this thread, it seems to me that for now it's best to keep both the pyproject.toml
version and the package.__version__
.
So my process is just to stop me accidentally forgetting to update one.
I've got a simple test that checks that those two are in alignment and fails if not.
import toml
from pathlib import Path
import my_package
def test_versions_are_in_sync():
"""Checks if the pyproject.toml and package.__init__.py __version__ are in sync."""
path = Path(__file__).resolve().parents[2] / "pyproject.toml"
pyproject = toml.loads(open(str(path)).read())
pyproject_version = pyproject["tool"]["poetry"]["version"]
package_init_version = my_package.__version__
assert package_init_version == pyproject_version
Seems to do the job without having to resort to some of the convoluted solutions above which may be hard to maintain
Here is my suggestion.
First, poetry new package
create __init__.py
as an empty file. In addition, create __version__.py
next to it. But the content is:
#v0.1.0
( its a hashtag. pun intended. xD ) Anyway,
poetry version
along with pyproject.toml
VERSION="0.1.0"; echo "#v$VERSION" > package/__version__.py
can bump version#v
will be the marker for the regular expression to search and find it even if the file contains any other content.The following code can be used inside setup.py
to obtain the version
#!/usr/bin/env python3
import re
from pathlib import Path
HERE = Path(__file__).parent
version_file = HERE / "CloudflareAPI/__version__.py"
version_file = version_file.resolve(strict=True).read_text()
version = re.search(r'#v(.+)', version_file).group(1)
print(version)
It can also be used in test files where the version is required.
(I do not feel like using importlib.metadata
)
Any thoughts?
I'm coming in here in Jan 2022, and I'm actually just looking for the canonical answer. Where is the single source of truth spot to put the version string (or where in the docs as poetry version doesn't seem to be the place).
I ran across this: https://pythonrepo.com/repo/tiangolo-poetry-version-plugin-python-package-dependency-management
which seems to show that others have this issue and are solving it with plugins but if I missed this a pointer in the right direction is appreciated.
I'm coming in here in Jan 2022, and I'm actually just looking for the canonical answer. Where is the single source of truth spot to put the version string (or where in the docs as poetry version doesn't seem to be the place).
I ran across this: https://pythonrepo.com/repo/tiangolo-poetry-version-plugin-python-package-dependency-management
which seems to show that others have this issue and are solving it with plugins but if I missed this a pointer in the right direction is appreciated.
I'd also like some clarification. I'm not sure what to follow. There are still several different plugins floating around:
I'm fine to use importlib.metadata
, so long as I can also use it for local packages I am developing (develop = true
). Has anyone tested this?
If you package your project, the version number is integrated into it. Then use
importlib_metadata
(since python 3.8 as build-inimportlib.metadata
) to access the version number during run time if needed.
Does this work for packages in development mode? Can you point me to tests?
Well FWIW in jsonvice I solved the problem by adding:
and this in the master source code file:
import pkg_resources # part of setuptools version = pkg_resources.require("jsonvice")[0].version
Now there is only one place the version is only specified in pyproject.toml and the string is available at runtime so the program can self-report its version number when queried.
I have published poetry-bumpversion plugin to update version in other files when poetry upgrades the package version. To update __version__
just install the plugin and add the following to your pyproject.toml
file.
[tool.poetry_bumpversion.file."your_package/__init__.py"]
FWIW I'm using this:
import importlib.metadata
__version__= importlib.metadata.version('<package name>')
Works fine 👍
I also switched to importlib.metadata
But I had to change my release workflow, before it was fully automated as I was able to choose in advance the version number during development.
My Take Away for importlib.metadata
Pros:
No more version number in files
No more required to bump/commit/push a new version during development or release workflow
Version is only available in git tags and inside package
relatively easy to automate for patches releases even with branch protection
Cons: hard to automated for minor and major release
Here is the github action snippet code to trigger a package publish for version 1.0.0 My Workflow
I use manually gh cli gh release create 1.0.0 --generate-notes
or directly from github.com (it generates the same event)
name: publish package
on:
push:
tags:
- '**'
jobs:
publish:
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v3
name: Checkout code
- uses: actions/setup-python@v4
name: Set up Python
with:
python-version: '3.10'
- name: Install tools
run: |
pip install --upgrade pip
pip install poetry
- name: Publish distribution 📦 to PyPI
run: |
poetry config repositories.pypi ${{ secrets.PYPI_REPOSITORY }}
poetry version ${{ github.ref_name }}
poetry publish --build -r pypi -u ${{ secrets.PYPI_PUBLISH_USERNAME }} -p ${{ secrets.PYPI_PUBLISH_TOKEN }}
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
I don't know if it's intended or not. A way to do that safely is to parse the root
__init__.py
, detect__version__
, back it up, extract the ast, bump the version string and replace it, then extract the new ast and compare the result. If both ast are the same, except for the version, the file semantics have been preserved. Otherwise, rollback the change and display an error message stating we can't bump the version safely.