search

qutip / qutip

QuTiP: Quantum Toolbox in Python
https://qutip.org
BSD 3-Clause "New" or "Revised" License
1.69k stars 635 forks source link

Continuous integration software: Travis CI, CirceCI, etc. #959

Closed nathanshammah closed 3 years ago

nathanshammah commented 5 years ago

This is post for discussion over whether it might be beneficial to consider additional continuous integration (CI) tools for QuTiP, beyond Travis CI.

Some open-source projects use multiple CI tools, possibly to do different tasks and test different configurations. For example by adding QuTiP to the list of projects using Sphinx for the documentation I noticed they use concurrently Travis CI, CircleCI and Appveyor.

Stackshare allows to compare multiple developers tools.

ajgpitch commented 5 years ago

I think Appveyor is what we need to set up in order to have our tests run on Windows. I don't think this is free. Needs more investigation. It would definitely be a very good thing to have tests run on Windows

abhisekupadhyaya commented 5 years ago

I have seen a lot of projects using CircleCI as it provides very fast builds, although it is not free. Moreover pep8speaks is a great app for automatically reviewing Python code style over Pull Requests.

nathanshammah commented 5 years ago

I am checking out Azure pipelines, which seems to have some (preferential?) integration with GitHub https://azure.microsoft.com/en-us/services/devops/pipelines/. Cc @tehruhn also for the previous comments.

jakelishman commented 3 years ago

Bumping an old topic with two major new considerations.

Number one: Travis is changing its pricing model for open source software, and generally it seems like it's going to become unfeasible for us. Over the past month or so, queue times on Linux builds have increased massively (from near-instant to up to 90 minutes), and the number of allowed concurrent builds has already limited the amount of testing we are able to do - we don't even have Windows tests because we couldn't fit them in. In around March 2021 we expect the old travis-ci.org that QuTiP is grandfathered onto to shut down completely, forcing us onto the new model on travis-ci.com.

Number two: in November 2019, GitHub released their own CI, GitHub Actions. This is (as best as I can tell) completely free for open source, public repositories like QuTiP, and offers access to a large number of Linux, macOS and Windows containers.

This completely free period of GitHub Actions might not last forever (you can imagine maybe they're just trying to drive adoption before increasing the cost), but especially with the new time pressure from Travis, this seems like the right choice for now. Our testing set up is fairly straightforward, so migrating to a new CI shouldn't be too difficult.

I just wrote (#1429) a GitHub Actions workflow to build and distribute binary releases of QuTiP on Linux/Mac/Windows, and I'm pretty convinced by the power of their CI. It's got an awful lot of flexibility with a lot of customisation so its initial learning curve is a little steep (or maybe I'm just not used to CI tools), but I'm impressed by what's available. I believe it comes with a degree of reusability as well; I think we can define a test as a single "action", and then reuse that in the PR-push workflow and the make-distribution workflow.

nathanshammah commented 3 years ago

+1 for GH for CI.

hodgestar commented 3 years ago

+1 for GH for CI from me too.

We can also have multiple ones if we need different CIs for special purposes (e.g. maybe one day we will want to run QuTiP on Arm64 devices), but let's start with one main one and go from there.