lda-project / lda

Topic modeling with latent Dirichlet allocation using Gibbs sampling
https://lda.readthedocs.io/
Mozilla Public License 2.0
1.24k stars 390 forks source link

Add GitHub Actions workflow #125

Closed severinsimmler closed 11 months ago

severinsimmler commented 11 months ago

So this is basically the last thing we need to build and publish the wheels. I added a GitHub Actions workflow that runs every time something is pushed to the master branch. This is basically a matter of taste, we could also configure it to run when someone pushes a specific tag – I'm fine with everything.

If an event triggered the workflow, it will set up a Windows, macOS and Ubuntu machine, install everything, build wheels with cibuildwheel and upload both the wheels and source dist to PyPI using Poetry. This requires the project owner's PyPI username (PYPI_USER) and password (PYPI_PW) to be available as environment variables (you can set these in the repository's settings page as secrets). Please also see the GitHub docs on using secrets in GitHub Actions: https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions

The builds on all three platforms should work as expected. I tested it with my fork (and the --dry-run option for poetry publish): https://github.com/severinsimmler/lda/actions/runs/6866391327

Since the logs of the mentioned build are only temporarily available (a few weeks I think?), this would've been published:

Linux

lda-2.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
lda-2.0.0-cp39-cp39-musllinux_1_1_x86_64.whl
lda-2.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
lda-2.0.0-cp310-cp310-musllinux_1_1_x86_64.whl
lda-2.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
lda-2.0.0-cp311-cp311-musllinux_1_1_x86_64.whl
lda-2.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
lda-2.0.0-cp312-cp312-musllinux_1_1_x86_64.whl

macOS

lda-2.0.0-cp39-cp39-macosx_12_0_x86_64.whl
lda-2.0.0-cp310-cp310-macosx_12_0_x86_64.whl
lda-2.0.0-cp311-cp311-macosx_12_0_x86_64.whl
lda-2.0.0-cp312-cp312-macosx_12_0_x86_64.whl

Windows

lda-2.0.0-cp39-cp39-win_amd64.whl
lda-2.0.0-cp310-cp310-win_amd64.whl
lda-2.0.0-cp311-cp311-win_amd64.whl
lda-2.0.0-cp312-cp312-win_amd64.whl

Source dist

lda-2.0.0.tar.gz

Having all this, building and publishing wheels for e.g. Python 3.13 next year should be super easy and straightforward: we just have to add 3.13 to the pyproject.toml and trigger the GH Actions build.

ariddell commented 11 months ago

One more thing: shouldn't tests be run before publishing?

ariddell commented 11 months ago

If moving the tests around and switching over to pytest is the easiest way to do this, I'm completely fine with that.

Thanks again for getting the source install working. This alone is an enormous advance.

severinsimmler commented 11 months ago

@ariddell, I removed the unused files and the dry run looked good: https://github.com/severinsimmler/lda/actions/runs/6946738382/job/18898933693

As mentioned above, tests run inside of cibuildwheel. If one fails, the whole build fails.

I also tried to update the docs, but not 100% sure about everything.

ariddell commented 11 months ago

This all looks great. Curious about how cibuildwheel runs the tests -- but I can look at that later.

severinsimmler commented 11 months ago

Awesome, thanks! Not sure what cibuildwheel's default test command is (see also https://cibuildwheel.readthedocs.io/en/stable/options/#test-command), but we just do what's configured in our pyproject.toml:

https://github.com/lda-project/lda/blob/aee65d31606a6944818cda32d6f151f347989781/pyproject.toml#L67-L68

ariddell commented 11 months ago

@severinsimmler thank you so much for this work. If you would like credit as an author of the package, please add your name to the .mailmap file in a PR. This is how I've been keeping track of significant contributions.