`skyplane init` error: `ModuleNotFoundError: No module named 'azure'`

[sarahwooders]

I pip installed Skyplane and ran skyplane init but immediately got an error:

(base) [ec2-user@ip-172-31-80-140 ~]$ skyplane init
Traceback (most recent call last):
  File "/opt/conda/bin/skyplane", line 5, in <module>
    from skyplane.cli.cli import app
  File "/opt/conda/lib/python3.9/site-packages/skyplane/cli/cli.py", line 16, in <module>
    import skyplane.cli.usage.client
  File "/opt/conda/lib/python3.9/site-packages/skyplane/cli/usage/client.py", line 9, in <module>
    from skyplane.replicate.replicator_client import TransferStats
  File "/opt/conda/lib/python3.9/site-packages/skyplane/replicate/replicator_client.py", line 20, in <module>
    from skyplane.compute.azure.azure_cloud_provider import AzureCloudProvider
  File "/opt/conda/lib/python3.9/site-packages/skyplane/compute/azure/azure_cloud_provider.py", line 17, in <module>
    from skyplane.compute.azure.azure_server import AzureServer
  File "/opt/conda/lib/python3.9/site-packages/skyplane/compute/azure/azure_server.py", line 3, in <module>
    import azure.core.exceptions
ModuleNotFoundError: No module named 'azure'

[sarahwooders]

Note I was able to fix this by running pip install azure.

[dbickson]

I was not able to instal azure with the following error (ubuntu 20.04, python3.8)

Downloading azure-5.0.0.zip (4.6 kB)
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3xxtn1qu/azure/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3xxtn1qu/azure/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-3xxtn1qu/azure/pip-egg-info
         cwd: /tmp/pip-install-3xxtn1qu/azure/
    Complete output (24 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-3xxtn1qu/azure/setup.py", line 60, in <module>
        raise RuntimeError(message)
    RuntimeError:

    Starting with v5.0.0, the 'azure' meta-package is deprecated and cannot be installed anymore.
    Please install the service specific packages prefixed by `azure` needed for your application.

    The complete list of available packages can be found at:
    https://aka.ms/azsdk/python/all

    Here's a non-exhaustive list of common packages:

    -  azure-mgmt-compute (https://pypi.python.org/pypi/azure-mgmt-compute) : Management of Virtual Machines, etc.
    -  azure-mgmt-storage (https://pypi.python.org/pypi/azure-mgmt-storage) : Management of storage accounts.
    -  azure-mgmt-resource (https://pypi.python.org/pypi/azure-mgmt-resource) : Generic package about Azure Resource Management (ARM)
    -  azure-keyvault-secrets (https://pypi.python.org/pypi/azure-keyvault-secrets) : Access to secrets in Key Vault
    -  azure-storage-blob (https://pypi.python.org/pypi/azure-storage-blob) : Access to blobs in storage accounts

    A more comprehensive discussion of the rationale for this decision can be found in the following issue:
    https://github.com/Azure/azure-sdk-for-python/issues/10646

    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

The package that did work for me was azure-mgmt-storage. And then I got another error

│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │             args = ()                                                                        │ │
│ │          err_msg = 'Cannot import google.auth. Install skyplane with gcp support: `pip       │ │
│ │                    install skypl'+9                                                          │ │
│ │               fn = <function GCPAuthentication.get_adc_credential at 0x7fa962c97820>         │ │
│ │           kwargs = {}                                                                        │ │
│ │           module = 'google.auth'                                                             │ │
│ │     module_split = ['google', 'auth']                                                        │ │
│ │          modules = ('google.auth',)                                                          │ │
│ │ modules_imported = []                                                                        │ │
│ │        pip_extra = 'gcp'                                                                     │ │
│ ╰─────────────────────────────────────────────────────────────────────────────────────────────```
which was solved with `pip install skyplane[gcp]`

[parasj]

Hi @dbickson, dependencies are bit messy and we are working to clean them up in https://github.com/skyplane-project/skyplane/pull/602. That PR will update the workflow to install azure-mgmt-storage when you run pip install skyplane[azure].

While we merge that PR, to install Azure dependencies manually, run: pip install --upgrade azure-identity azure-mgmt-authorization azure-mgmt-compute azure-mgmt-network azure-mgmt-resource azure-mgmt-storage azure-mgmt-subscription azure-storage-blob

[dbickson]

Hi @parasj since I am not using Azure anyway I suggest to do a delayed import only when someone asks for Azure config. (The error appeared before init is asking to configure Azure usage).

[cjrd]

I had the same issue, you can work around for now with

pip install azure-core

[parasj]

I pushed a fix based on @dbickson's suggestion to lazily import Azure via dependency injection. It seems to fix the issue in a clean environment (@sarahwooders tested this). I'm cutting a new release to integrate this new bugfix by end of day. Thanks for helping chase down the root case of this issue.

https://github.com/skyplane-project/skyplane/pull/613

[phi-line]

I think this issue can be closed as per https://github.com/skyplane-project/skyplane/pull/613