Migrate the metadata into `PEP 621`-compliant `pyproject.toml`.

[KOLANICH]

278 is a prereq

[Michael-E-Rose]

What's wrong with the cfg files?

[KOLANICH]

What's wrong with the cfg files?

It depends on your PoV. Mine is such:

1. I was an early adopter of setup.cfg in my projects. a. When it was available, I had migrated to them. b. When it had become possible to eliminate setup.py (when PEP 517 allowed to specify setuptools, build (that time it lived within pep517 package) allowed to trigger build and setuptools got support of setup.py-less projects), I had also done that.
2. From my experience: in order to deal with setup.cfg properly, one has to use the parser from within setuptools. Other parsers can parse it insorrectly, setup.cfg is not just serialization format, syntax of each field cannot be any, instead commas, semicolons and line breaks have different meanings in different fields. And there is no se4ializer for this custom format at all.
3. That's why it was standardized that pyproject.toml should be used. It is TOML, that is kinda a de-facto standard. There are different impls to both parse and serialize with it in different langs.
4. PEP 517 and PEP 518 have resulted into a dirty mess of build systems incompatible to each other. flit_core, hatchling, poetry_core, pdm, maturin (for Rust projects, a terrible bloatware).
5. to at least partially clean that mess PEP 621 was created. It is a unified format of metadata for the most of backends. IMHO it should have been specified before opening the Pandora box with PEP 517.

So the advantages are:

• TOML;
• mostly not specific to build backend (PEP 621 doesn't specify everything, now these parts are backend-specific and live in tool.setuptools, since the backend we use is setuptools).

[KOLANICH]

I'm making a package manager getting the packages from source code repositories. Though currently I have the code to extract metadata from all setup.cfg, pyproject.toml and setup.py, in long run I'd like to get rid of all the code except the one dealing with pyproject.toml. But in order to do it we need to migrate packages to pyproject.toml.

[Michael-E-Rose]

I'm all for staying up-to-date with PEP. Shame though that we rely on setup.cfg because of pbr. I'll resolve the merge conflict tomorrow.

[KOLANICH]

What are the advantages of pbr over the more popular build backends (setuptools, hatchling)? All of them support PEP 621 and plugins.

[Michael-E-Rose]

pbr still works with setuptools, but it's less work for me. My biggest advantage (that I still remember) was that I could versionize via commit message. With setuptools I still had to update the version number myself in setup.cfg (or setup.py, can't remember). Might be there were other advtanges, but I forgot them.

[KOLANICH]

With setuptools I still had to update the version number myself in setup.cfg (or setup.py, can't remember).

One can use setuptools_scm, this plugin is dedicated for that feature. Supports pure pyproject.toml-based workflow.

[Michael-E-Rose]

If we can replace pbr then users have to install one package less. That's good.

How would the process look like with alternatives?

Right now I put a version tag to a commit, then I create the tarball (while I update the change log myself), then I upload to PyPI.

[KOLANICH]

If we can replace pbr then users have to install one package less.

Not quite less, they will have to install setuptools_scm insttead (in fact build by default automatically installs the needed packages, unless a user opts out of it (that is recommended by me, because build uses cargo cult "build isolation" by making a venv and installing build deps freshly into it every time, which is IMHO waste of time, traffic and energy, and also may be a security issue) ). But it'd allow to get rid of unneeded legacy files.

How would the process look like with alternatives? Right now I put a version tag to a commit, then I create the tarball (while I update the change log myself), then I upload to PyPI.

The same, though I don't understand why you create a tarball instead of creating an sdist (which is a tarball, but with metadata for PyPI).

Also, it makes sense to upload wheels to PyPI.

[KOLANICH]

Also, it seems you use the feature of deps autodiscovery.

--- ./dist/a    2023-02-23 21:38:30.000000000 +0300
+++ ./dist/b    2023-02-23 21:38:30.000000000 +0300
@@ -2,3 +2,3 @@
 Name: pybliometrics
-Version: 3.4.1.dev23
+Version: 3.4.1.dev23+gba903b5.d20230223
 Summary: Python-based API-Wrapper to access Scopus
@@ -26,6 +26,3 @@
 License-File: LICENSE
-Requires-Dist: pbr
-Requires-Dist: requests
-Requires-Dist: simplejson
-Requires-Dist: tqdm
+License-File: AUTHORS

Also it generates changelog and authors list. I guess we should keep pbr for now.

[Michael-E-Rose]

When I use pbr, I suppress changelog and authors list in the command. Because the latter is fixed and I prefer to do the former manually. This is the command I use: SKIP_GENERATE_AUTHORS=1 SKIP_WRITE_GIT_CHANGELOG=1 python3 setup.py sdist - and then I upload the tar.gz file. It's in /dist. I thought it's a tarball because of the file ending, but it seems to be the thing that PyPI wants.

I am open to replacing pbr with setuptools_scm, if that allows a file structure that adheres to current standards.

[KOLANICH]

I thought it's a tarball because of the file ending

It is a tarball, but with an additional metadata dir.

but it seems to be the thing that PyPI wants.

Well, it is recommended for pretty a long time to upload both sdists and wheels (bdist_wheel) on PyPI. This way users can skip build step and use the prebuilt wheels. For build it is recommended by PyPA that your building command look like python3 -m build, it'd build both sdist and then a wheel from the sdist. If you want to speed-up things and reduce overhead, then it should look like python -m build -nwsx (w builds a wheel, s builds an sdist, everything directly from the repo this way, n disables cargo-cult "build isolation" and uses system-installed deps (don't forget to keep them up to date), x disables deps resolver (so build won't install anything without your explicit consent))

[Michael-E-Rose]

I switched entirely to setuptools_scm and dumped pbr. I think your suggested toml was sufficient (#284 ); I took another section as per your template. (And of course I had to remove some pbr and versioning legacy code.)

[Michael-E-Rose]

The move away from pbr cause more troubles than anticipated :/ While I was able to fix most things ex-post, a big problem with the automatic build on readthedocs fails, see https://stackoverflow.com/questions/75922593/sphinx-readthedocs-and-package-version. Do you have an idea?

[KOLANICH]

I consider the answer by Mikko Ohtamaa as correct in general (you need installation (in fact you can avoid it, you only need metadata dir to be generated, just building a wheel with disabled isolation (python3 -m build -nw) should be enough for it, but for the Read The Docs pipeline it is easier just to install the package) in order importlib_metadata to work), but instead of the link he has given, where there is no installation of the package itself (only its deps), you need to install the package itself.

- method: pip
  path: .

[KOLANICH]

Also, though it should have no effect on the problem you mention, it is recommended to upgrade os: ubuntu-22.04 from os: ubuntu-20.04 it now has.

[Michael-E-Rose]

Now I simply solved the problem by using pbr for Sphinx and ReadTheDocs.