search

qiboteam / qibotn

The tensor-network translation module for Qibo.
https://qibo.science
Apache License 2.0
3 stars 1 forks source link

Add wheels deploying workflow #43

Closed alecandido closed 4 months ago

alecandido commented 4 months ago

If we're aiming for a release, we should also make sure to release something.

alecandido commented 4 months ago

Since we're in an emergency situation, I dropped the generation of wheels for macos and windows, since (at least macos) has been proven to be incompatible with cuquantum.

It would be nice to have at least Quimb support for all platforms, but I'd face the problem after the release.

(I had a look to Qibojit, but there is no valuable solution in there: simply, the Cuquantum and CuPy dependencies are undeclared - they are only in a suitable Poetry group in Qibo, but it's definitely the wrong place) https://github.com/qiboteam/qibojit/blob/5af642977fd66a4268608fbe2986e55160a88fe4/pyproject.toml#L22-L27

alecandido commented 4 months ago

In order to install the dependencies, the cupy package has to build from source, and this can happen only in an environment where CUDA is available. https://github.com/qiboteam/qibotn/actions/runs/8121865123/job/22200696123?pr=43#step:9:126

That would make all workflows failing in the GitHub CI, so I used a pre-built wheel, that can be installed in any supported platform (i.e. Linux), and the CUDA libraries are just dynamically discovered (so, if you don't run CUDA code, their absence won't be discovered as well).

To do this, I had to pick a given CUDA version (at least major), and I went for 12x. If you have any argument for which 11x is preferable, then just let me know.

liweintu commented 4 months ago

Since we're in an emergency situation, I dropped the generation of wheels for macos and windows, since (at least macos) has been proven to be incompatible with cuquantum.

It would be nice to have at least Quimb support for all platforms, but I'd face the problem after the release.

(I had a look to Qibojit, but there is no valuable solution in there: simply, the Cuquantum and CuPy dependencies are undeclared - they are only in a suitable Poetry group in Qibo, but it's definitely the wrong place) https://github.com/qiboteam/qibojit/blob/5af642977fd66a4268608fbe2986e55160a88fe4/pyproject.toml#L22-L27

Could we use the CUDA availability from a preceding check job to guide the build job, similar to the workflow in rules.yml? I assume a new job for building a quimb-only package might be needed.

alecandido commented 4 months ago

Could we use the CUDA availability from a preceding check job to guide the build job, similar to the workflow in rules.yml? I assume a new job for building a quimb-only package might be needed.

We could, but with no benefit in the short term: the purpose of that check right now is to prevent tests to run at all. If we prevent build to run, we'll be left with no released wheels.

liweintu commented 4 months ago

We could, but with no benefit in the short term: the purpose of that check right now is to prevent tests to run at all. If we prevent build to run, we'll be left with no released wheels.

That's true as we only have one build job for both quimb and cuquantum. I was thinking if a new job (or step) could be added, such that when CUDA availability is detected as false, it could build a quimb-only package.

alecandido commented 4 months ago

That's true as we only have one build job for both quimb and cuquantum. I was thinking if a new job (or step) could be added, such that when CUDA availability is detected as false, it could build a quimb-only package.

Yes, but I'd try to design anything more complex after the release, to minimize the risk of having no release at all by Tuesday.

liweintu commented 4 months ago

Yes, but I'd try to design anything more complex after the release, to minimize the risk of having no release at all by Tuesday.

Yup, that makes sense. Proceed to approve and merge this PR.