Publishing to both TestPyPI and PyPI using GitHub Actions

[tomaroberts]

Hi there,

I've been playing around with various dynamic versioning tools for a couple of days now, in particular to use in conjunction with GitHub Actions, TestPyPi and PyPI. I think I'm going to go ahead with this or regular dunamai.

What I can't quite get my head around is the best approach to publishing dynamically on both TestPyPI for dev work and PyPI for releases.

I'll explain my current thinking below, but I also have a feeling I have a fundamental misunderstanding of best practices around tagging and releases, so please feel free to correct me if what I'm about to describe is stupid...

My intention is that:

• With every repo push, my package is published to TestPyPI using a GHA
  • Currently I'm using format 0.1.2.dev5, 0.1.2.dev6, etc.
  • I've got this working in a GHA (currently using setuptools-scm, but I'm going to switch to this – in principle this works though). Every commit increments the devX number and thus this results in unique versions on TestPyPI.
• Come release time, I want the package published to PyPI via GHA
  • desired format: 0.1.2
  • My trouble here is that I don't understand how to convert/push, say, version 0.1.2.dev22 to PyPI as simply 0.1.2. I can push to PyPI with 0.1.2.dev22, but that is just going to be weird having a history of say 0.1.1.dev16, 0.1.2.dev22, 0.1.3.dev8, and so on.

What am I missing here? I hope this makes sense. Happy to expand further.

Many thanks!

[mtkennerly]

Hi! I think you're on the right track here.

Every commit increments the devX number and thus this results in unique versions on TestPyPI.

I'd recommend also adding +{commit} because two branches that branched from the same commit would have the same distances.

I don't understand how to convert/push, say, version 0.1.2.dev22 to PyPI as simply 0.1.2

Let's say you start by releasing 0.1.0 (tagged as v0.1.0) and then make 5 commits.

If you use the dev segment without post, you'll get 0.1.0.dev5, which technically should be a pre-release of 0.1.0, but that's not correct since 0.1.0 was released first. If you don't want the post segment, then I'd recommend using the bump config option, which will give you 0.1.1.dev5 in the default format.

Then, to convert 0.1.1.dev5 to 0.1.1, all you have to do is make a v0.1.1 tag. In the default format, when the distance is 0 (i.e., we're on a tagged commit), it won't output the dev segment or other metadata. The next commit will generate 0.1.2.dev1, and the cycle repeats.

If you're using a custom format, then you'd need to use the format-jinja option so that you can make it conditional on the distance, but I think the default format with the bump option will work best here.

[tomaroberts]

Thanks for the quick reply!

My understanding was that PyPI/TestPyPI are PEP440 so don't allow git commit numbers in release names, unless I'm mistaken?

Then, to convert 0.1.1.dev5 to 0.1.1, all you have to do is make a v0.1.1 tag. In the default format, when the distance is 0 (i.e., we're on a tagged commit), it won't output the dev segment or other metadata. The next commit will generate 0.1.2.dev1, and the cycle repeats.

I see – makes sense now I've played run git tag v0.1.1 locally, thanks. Howeverrr, when I push to my repo, the tag is not pushed? i.e.:

Should I be setting tags in GitHub rather than locally?

[tomaroberts]

Separate question: the README is great but one thing that I'm unsure about is where it says:

Poetry's typical version setting is still required in [tool.poetry], but you are encouraged to use version = "0.0.0" as a standard placeholder.

Can I clarify: this should always be 0.0.0 and it's not actually used by this plugin?

[mtkennerly]

My understanding was that PyPI/TestPyPI are PEP440 so don't allow git commit numbers in release names, unless I'm mistaken?

PEP 440 allows a "local version identifier" separated by a plus sign: https://peps.python.org/pep-0440/#local-version-identifiers

Howeverrr, when I push to my repo, the tag is not pushed?

You probably just need to run git push --tags.

Can I clarify: this should always be 0.0.0 and it's not actually used by this plugin?

The plugin doesn't care about the value, but Poetry itself still expects the field to exist. The exact placeholder value doesn't matter, so 0.0.0, 1.2.3, 99.99.99, etc, would all be fine, but I like 0.0.0 since it's not a version that would ever normally be released.

[tomaroberts]

Awesome, thank you. Didn't realise you push tags... always learning!