Set up GitHub Action + PyPi

[maximilianvz]

I'd like to propose setting up a GitHub Actions workflow to upload AtomDB to PyPi using Twine whenever a release is created. This would require us to create a PyPI API token and add this to the repository's secrets. If I am to do this, I need to know 2 things:

1. Is there a PyPI account through which all QC-Devs packages are published? If so, I would need access to it to publish the package initially. From there, I can generate the PyPI API token. Otherwise, I can make my own account.
2. May I have the correct permissions in this repository to modify GitHub secrets? If not, someone with said permissions will need to handle that. However, it seems a very simple task, so no worries either way.

[PaulWAyers]

@maximilianvz @FarnazH there is a PyPi account, and I think that @msricher @marco-2023 @FarnazH @tovrstra have the "secrets". I'm not sure what level of permissions you need, but I've set you up as a "maintainer" so that should suffice I think.

[maximilianvz]

Thanks, @PaulWAyers. Setting me as a maintainer is sufficient to grant me access to secrets. To use GitHub actions, someone with access to the PyPi account will have to obtain the necessary API token, but there are some prerequisite steps that I will have to investigate before this can be done.

[gabrielasd]

Hi @maximilianvz were you able to add AtomDB to PyPi?

[maximilianvz]

Hey, @gabrielasd. I was waiting for API changes to be finalized before pushing to PyPi, and now the API appears finished. Unfortunately, I don't have the bandwidth to do this presently, so either someone else should take over or I will have to get to it in a couple of weeks.

[tovrstra]

I'll try to give some advice to facilitate the packaging.

These days, PyPI allows multiple package owners for a single PyPI package, which is much nicer than trying to share a password for a bot account that everyone uses. (Ours is named theochem on PyPI, but we can move away from it. I already did for IOData.)

So practically, you can follow the instructions from the Python Packaging User Guid to configure trusted publishing for the qc-atomdb package in your account, both for the official PyPI and the testing platform. This way, you don't need to set up tokens.

The workflow file in that guide is outdated. You can copy the following workflow: https://github.com/theochem/iodata/blob/main/.github/workflows/release.yaml and modify the line with PYPI_NAME to have qc-atomdb instead of qc-iodata.

I think you should also add a section to pyproject.toml, something like:

[tool.setuptools_scm]
write_to = "atomdb/_version.py"
version_scheme = "post-release"
local_scheme = "no-local-version"

and remove the atomdb/version.py file from the Git repo. You can add atomdb/_version.py to your .gitignore because it gets generated when needed. This can be combined with the following import in atomdb/__init__.py (above all other imports):

try:
    from ._version import __version__, __version_tuple__
except ImportError:
    __version__ = "0.0.0.post0"
    __version_tuple__ = (0, 0, 0, "a-dev")

Before making a release with a tag, I'd suggest adding a changelog file. The following guide is quite helpful: https://keepachangelog.com/en/1.1.0/

Once this is all set up, you can make a real release by tagging a commit with vX.Y.Z and running git push origin --tags. For all other commits, test releases are uploaded to test.pypi.org, so you're sure it works well by the time you make a real release. (test.pypi.org sometimes fails with a 503, which can be ignored.)

Once you have made a first release of a package, you can invite other accounts to become co-owners on PyPI, as to increase the bus factor. It would be good to do this for both testing and official PyPI platforms.

If you have any questions with all this, please let me know. Good luck!

[gabrielasd]

Hey, @gabrielasd. I was waiting for API changes to be finalized before pushing to PyPi, and now the API appears finished. Unfortunately, I don't have the bandwidth to do this presently, so either someone else should take over or I will have to get to it in a couple of weeks.

@maximilianvz no problem, it can wait until you are back.

[gabrielasd]

@FanwangM, I think this line in the pypi_release file has a typo and should point to atomdb repo: https://github.com/theochem/AtomDB/blob/18f3c936aaf774678423dd12d8ceb6556b2dcfcd/.github/workflows/pypi_release.yaml#L40

[FanwangM]

Thanks for noticing this. This is not a typo, but to install the dependency grid manually as we don't have PyPI release of grid yet (on the way). I just had a discussion with @marco-2023 and this will be addressed once we have the PyPI release of grid. Also, we will have the optional dependency install enabled soon. @gabrielasd

[gabrielasd]

Ah! thanks for the clarification @FanwangM and for taking the lead with this issue 👍

[FanwangM]

Can I close this issue, given we have the GitHub Actions and PyPI setup already? @gabrielasd @PaulWAyers @msricher