Please help maintain the Azure-cli conda packages

[blackbox-tech]

The azure-cli packages on conda-forge and bioconda are over a year out of date. As a result, Azure is not really usable with conda (for example only the legacy azure cli, rather than the az cli is available into a conda environment and not with python 3.8 or 3.9). You can hack it by using pip to install into a conda environment but this is dangerous and can break things.

If Microsoft wants Azure to reach the scientific and quantitative communities it should really do more help to maintain the packages that serve its products to these data science practitioners.

I've also tried manually building the azure-cli package with conda skeleton, but the recipe that is automatically generated is not understood by conda so the build fails.

$ conda skeleton pypi azure-cli

[yungezz]

hi @fengzhou-msft could you pls help to have a look? thanks

[ltalirz]

I agree that being able to install the azure-cli via conda from conda-forge would be a huge plus for the scientific community

Nowadays, conda-forge feedstock updates are quite low maintenance - PRs are opened automatically and most of the time one just needs to merge them. Having a maintainer from the azure team on board would be very helpful here.

Currently, the automatic builds fail https://github.com/conda-forge/azure-cli-core-feedstock/pulls and the logs are already gone, so I'm opening a new PR to see why the builds are failing https://github.com/conda-forge/azure-cli-core-feedstock/pull/14

P.S. I just noticed that one can install the azure-cli also from PyPI. The installation instructions for MacOS only mention the homebrew route. Is there a reason why pip install azure-cli is not listed here?

[jiasli]

We release Azure CLI on PyPI and we do have a script to install Azure CLI with pip.

You may also manually use

pip install azure-cli

However, we frequently receive packaging/environment issues of Azure CLI caused by pip installation: https://github.com/Azure/azure-cli/issues?q=is%3Aissue+%22Installer%3A+PIP%22

Azure CLI has lots of dependencies and the user's environment can often have conflicting dependencies (such as https://github.com/Azure/azure-cli/issues/15530). That's why we don't recommend installing with pip. Please use the official installation packages whenever possible.

[ltalirz]

thanks for the explanation @jiasli ! Yes, I noticed the rather narrow dependency specifications of the azure-cli package. I just created a new conda environment and then ran pip install azure-cli in there; perhaps that is a model that could work also for others.

[jiasli]

@ltalirz, I am glad to know it works for you.

I noticed the rather narrow dependency specifications of the azure-cli package.

Exactly. Azure CLI has a complex dependency network on various Python SDKs (including Track 1 + Track 2) and other underlying libraries. We also need to frequently address Common Vulnerabilities and Exposures (CVE) by bumping the minimum versions required to meet our security enforcement.

Here is another example that requests==2.22.0 required by customers in their requirements.txt can cause tricky endless backtracking when installing Azure CLI using pip with the new resolver: https://github.com/Azure/azure-cli/issues/18584#issuecomment-893944036.

[jiasli]

Temporarily closing this issue. We can reopen it if there are more votes.

[majidaldo]

If it has complex deps, then it should be installed in Python's venv. However, I still have to fiddle with the right Python version.

[majidaldo]

fwiw, conda installing jsmin first, then pip installing azure cli seems to work well.

[jiasli]

@majidaldo, are you referring to the jsmin issue https://github.com/Azure/azure-cli/issues/19468? Or are you seeing any errors? You may create a new issue with detailed description of your problem.

[majidaldo]

@majidaldo, are you referring to the jsmin issue #19468? Or are you seeing any errors? You may create a new issue with detailed description of your problem.

Oh I see. Thanks.

[AronT-TLV]

In this ticket I explain why on Mac OS X is not a sufficiently reliable way to install Azure CLI. Essentially the Python version brew installs may not be compatible with teh current bersion of the Azure CLI. It would be very nice if there was a conda version.

[ltalirz]

@AronT-TLV What happens if you pip-install azure-cli in a conda environment that thas an appropriate version of python?

P.S. I once had a quick look into adding azure-cli to conda-forge but it looks like this would involve a bit of work; see discussion in https://github.com/conda-forge/azure-cli-core-feedstock/pull/14

[majidaldo]

@AronT-TLV What happens if you pip-install azure-cli in a conda environment that thas an appropriate version of python?

P.S. I once had a quick look into adding azure-cli to conda-forge but it looks like this would involve a bit of work; see discussion in conda-forge/azure-cli-core-feedstock#14

i do this all the time. i keep az in its own environment though.. away from the env that has my application code. i call az through a wrapper from the application env.

[jiasli]

@AronT-TLV, your problem with Homebrew https://github.com/Azure/azure-cli/issues/20461 is a known issue #20348.

Actually, you may use pip install azure-cli on any platform with a fixed Python version.

[AronT-TLV]

Thanks @jiasli Just wanted to note however you yourself say: "That's why we don't recommend installing with pip" I plan to continue to use brew, just leave myself the option to run it in a Conda environment with a fixed previous version of Python, which is a very low overhead solution. I do still think it might be useful to add that as troubleshooting guidance in the doco.

[jiasli]

That's why we don't recommend installing with pip

pip installation is official supported. It is just people always install azure-cli with other pip packages in the same virtual environment, thus breaking azure-cli's dependencies. That's why I said "we don't recommend installing with pip".

I do still think it might be useful to add that as troubleshooting guidance in the doco.

Thanks for the suggestion. We will consider adding this instruction or perhaps replacing https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=script, which is too complex/automated and leaves users with few customization options.

[h-vetinari]

I know how to do this, but it's most likely 10-20 hours or more of initial work (not counting the reviewers in staged-recipes), and then probably 2-3h a months to keep all the many tiny packages up-to-date. I'd like to have azure-* in conda-forge (the most integrated, complete, and also license-free channel), but this is too much & too one-sided for volunteer work.

I doubt that anyone would want to fund such an effort, but if so, feel free to let me know. 🙃

[crazy4pi314]

Wanted to bump this issue, just came from tracing some bugs in the Dask Azure integration which does not work from conda (their recommended environment manager.) As a primarily windows scientific Python dev, conda is often required to setup tools as there are dependencies outside of Python that I need to setup for a project (ex. llvm, rust, etc). The paths mentioned above of installing pip and then using pip to install the Azure tools works, but it makes installation longer and dependency management split between conda and pip. So not a hard blocker, but would mean a lot I think to the scipy community (especially projects looking to add/maintain azure support (https://github.com/conda-forge/dask-cloudprovider-feedstock/pull/9#discussion_r552289241)) to have a conda package maintained for Azure configuration.

[majidaldo]

Wanted to bump this issue, just came from tracing some bugs in the Dask Azure integration which does not work from conda (their recommended environment manager.) As a primarily windows scientific Python dev, conda is often required to setup tools as there are dependencies outside of Python that I need to setup for a project (ex. llvm, rust, etc). The paths mentioned above of installing pip and then using pip to install the Azure tools works, but it makes installation longer and dependency management split between conda and pip. So not a hard blocker, but would mean a lot I think to the scipy community (especially projects looking to add/maintain azure support (conda-forge/dask-cloudprovider-feedstock#9 (comment))) to have a conda package maintained for Azure configuration.

may i introduce you to https://github.com/pnnl/hydraconda ? i side step all this by creating contained installations of az cli that connect to your 'main'/'work' env via wrappers. the azure installation is quite complex pulling an insane amount of deps.