ucx-py causing issues

[rbeucher]

Hi @dsroberts

I am having trouble setting up an ESMValTool environment here: https://github.com/ACCESS-NRI/MED-condaenv/pull/66

Error message is:

error    libmamba Could not solve for environment specs
    The following packages are incompatible
    �~T~\�~T~@ python_abi 3.11 4_cp311 is installable and it requires
    �~T~B  �~T~T�~T~@ python 3.11.* *_cpython, which can be installed;
    �~T~T�~T~@ ucx-py is not installable because it requires
       �~T~T�~T~@ python >=3.10,<3.11.0a0 , which conflicts with any installable versions previously reported.
critical libmamba Could not solve for environment specs

I get ucx-py from the coecms conda channel.... Is there something we could do to help with that?

R

[dsroberts]

Hi @rbeucher

We could probably do that. At this stage ucx-py is only provided by rapidsai for Python 3.9 and 3.10, though I think that's a reflection of the fact that most users will build ucx-py from source. It looks like we're a bit behind in ucx-py releases anyway, I may as well do another one for ucx-py/0.36.00, and build it against ucx/1.15.0 now that that version is available on Gadi & conda-forge. I'll try a Python 3.11 version for you guys, though I'm not sure what to do if it doesn't work...

[rbeucher]

Thanks @dsroberts !

[rbeucher]

It would be good to know the process so we can do it next time @dsroberts

[dsroberts]

Yep, sure. I'm at SCAsia so won't be able to look at this until later in the week. You don't have to follow our approach here either, you could build from source or add the rapidsai conda channel.

[dsroberts]

OK, new ucx-py packages are now uploaded. I've done ucx-py 0.36.00 for UCX 1.14.0 and 1.15.0 for Python 3.10 and 3.11. A quick test with a new environment seems to confirm that it'll pick the right variant of ucx-py depending on which version of UCX you ask for. As I said before though, this is somewhat outside the bounds of what rapidsai supports. As for doing this yourself, we maintain a fork of ucx-py here https://github.com/coecms/ucx-py that has a few customisations to the build setup. The main branch on the upstream is kept up to date with releases, and we have a couple of branches off of that (main.cms.ucx1.14.0 and main.cms.ucx1.15.0) that we build from. If you want a variant outside of what we provide, you'll need to upload it to the accessnri conda channel.