search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.16k stars 5.48k forks source link

salt-cloud 2015.5.0 azure provider could not be loaded #27980

Closed rayba closed 8 years ago

rayba commented 9 years ago

We have configured both an AWS and Azure provider for a salt-cloud POC.

The AWS provider works but we receive the below error for Azure when running any command such as: 

salt-cloud --list-providers

we receive the error: _The cloud driver, 'azure', configured under the 'my-azure-config' cloud provider alias, could not be loaded. Please check your provider configuration files and ensure all required dependencies are installed for the 'azure' driver. In rare cases, this could indicate the 'azure.get_configuredprovider()' function could not be found. Removing 'azure' from the available providers list

We are working from the instructions at: https://docs.saltstack.com/en/2015.5/topics/cloud/azure.html and have replaced the subscription_id with our own Azure subscription id. The relevant parts of salt-cloud directory structure under /etc/salt looks like this:

.
├── azure.cer
├── azure.pem
├── cloud
    ├── cloud.conf.d
    ├── cloud.deploy.d
├── cloud.maps.d
│   └── cloud.map
├── cloud.profiles.d
│   ├── azure.profile.conf
│   └── ec2.profile.conf
├── cloud.providers.d
│   ├── azure.provider.conf
│   └── ec2.provider.conf

our azure.provider.conf is this:

my-azure-config:
  provider: azure
  subscription_id: (our subscription id)
  certificate_path: /etc/salt/azure.pem
  requests_lib: True
  minion:
    master: saltmaster.(our domain).net

  management_host: management.core.windows.net

our azure.profile.conf is this:

azure-ubuntu:
  provider: azure
  image: 'b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-12_04_3-LTS-amd64-server-20131003-en-us-30GB'
  size: Small
  location: 'East US'
  ssh_username: azureuser
  ssh_password: verybadpass
  slot: production
  media_link: 'http://portalvhdabcdefghijklmn.blob.core.windows.net/vhds'

We have installed all dependencies (Azure python SDK, OpenSSL) and our python version is 2.7.10.

techhat commented 9 years ago

Hi @rayba. Could you please post the output from salt-cloud -l debug --list-sizes my-azure-config and salt-cloud --versions-report? Also, which version of requests are you running?

rayba commented 9 years ago

Python requests version: 1.2.3

salt-cloud -l debug --list-sizes my-azure-config

[DEBUG   ] Reading configuration from /etc/salt/cloud
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: ec2-52-20-57-248.compute-1.amazonaws.com
[DEBUG   ] Missing configuration file: /etc/salt/cloud.providers
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/azure.provider.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/azure.provider.conf
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/ec2.provider.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/ec2.provider.conf
[DEBUG   ] Missing configuration file: /etc/salt/cloud.profiles
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/azure.profile.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/azure.profile.conf
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/ec2.profile.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/ec2.profile.conf
[DEBUG   ] Configuration file path: /etc/salt/cloud
[INFO    ] salt-cloud starting
[DEBUG   ] Could not LazyLoad parallels.avail_sizes
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy
[DEBUG   ] Could not LazyLoad saltify.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.avail_images
[DEBUG   ] Could not LazyLoad saltify.avail_locations
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Could not LazyLoad azure.get_configured_provider
[WARNING ] The cloud driver, 'azure', configured under the 'my-azure-config' cloud provider alias, could not be loaded. Please check your provider configuration files and ensure all required dependencies are installed for the 'azure' driver.
In rare cases, this could indicate the 'azure.get_configured_provider()' function could not be found.
Removing 'azure' from the available providers list
Error: There was an error listing sizes: No cloud providers matched 'my-azure-config'. Available selections: crooms-ec2

salt-cloud --versions-report

            Salt: 2015.5.5
          Python: 2.6.9 (unknown, Apr  1 2015, 18:16:00)
          Jinja2: 2.7.2
        M2Crypto: 0.21.1
  msgpack-python: 0.4.6
    msgpack-pure: Not Installed
        pycrypto: 2.6.1
         libnacl: Not Installed
          PyYAML: 3.10
           ioflo: Not Installed
           PyZMQ: 14.3.1
            RAET: Not Installed
             ZMQ: 3.2.5
            Mako: Not Installed
         Tornado: Not Installed
         timelib: Not Installed
        dateutil: 2.4.2
 Apache Libcloud: 0.18.0
techhat commented 9 years ago

It looks like on 2015.5.x, if your provider is configured incorrectly, you should be getting:

[WARNING ] The cloud driver, 'azure', configured under the 'my-azure-config' cloud provider alias is not properly configured. Removing it from the available providers list

Yours isn't doing that, so I'm thinking it's a dependency issue. Could you please post the output from python -c 'import azure; print azure.__version__'?

Also, which OS are you running?

rayba commented 9 years ago

OS: Amazon Linux AMI release 2015.09

I'm getting this error:

> python -c 'import azure; print azure.__version__'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'module' object has no attribute '__version__'
techhat commented 9 years ago

That sounds like the problem: the Azure SDK seems to have a problem. Where did you install it from, and what version?

rallytime commented 9 years ago

I think that's coming from the new Azure SDK. I vaguely remember another issue where the new Azure SDK doesn't have a __version__ attribute. Let me see if I can find it.

rallytime commented 9 years ago

Ah, here it is: https://github.com/saltstack/salt/issues/26863

techhat commented 9 years ago

So @rayba it sounds like python -c 'import azure; print azure.common.__version__' should tell you 1.0.0 then. Could you please confirm?

rayba commented 9 years ago

Is there any way to use a prior version of the Azure SDK to allow us to complete our POC? By the way very impressed with the Salt community, it's a big part of why we chose Salt over other products. 

> python -c 'import azure; print azure.common.__version__'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'module' object has no attribute 'common'
techhat commented 9 years ago

Sorry, should be: python -c 'import azure.common; print azure.common.__version__'.

techhat commented 9 years ago 
pip install -Iv https://pypi.python.org/packages/source/a/azure/azure-0.6.zip#md5=5472666ee734defa880d82a512343828
techhat commented 9 years ago

Sorry, try this (for version 0.10.2):

pip install -Iv https://pypi.python.org/packages/source/a/azure/azure-0.10.2.zip#md5=8eaa0f8e649b21b6527a5ee801cef33a
rayba commented 9 years ago

I'm getting this error (for 0.10.2) install:

InstallationError: Command "/usr/bin/python2.7 -c "import setuptools, tokenize;__file__
='/tmp/pip-build-soP63s/cffi/setup.py';
exec(compile(getattr(tokenize, 'open', open)(__file__).
read().replace('\r\n', '\n'), __file__, 'exec'))" 
install --record /tmp/pip-zodX6f-record/install-record.txt 
--single-version-externally-managed --compile" failed 
with error code 1 in /tmp/pip-build-soP63s/cffi
techhat commented 9 years ago

Did you pip uninstall azure first?

rayba commented 9 years ago

just did, same error. Thanks.

jliu70 commented 9 years ago

I'm getting a similar error.

[root@ans01 salt]# salt-cloud --versions
/usr/lib64/python2.6/site-packages/Crypto/Util/randpool.py:40: RandomPool_DeprecationWarning: This application uses RandomPool, which is BROKEN in older release
s.  See http://www.pycrypto.org/randpool-broken
  RandomPool_DeprecationWarning)
Salt Version:
            Salt: 2015.8.1

Dependency Versions:
 Apache Libcloud: 0.18.0
          Jinja2: 2.2.1
        M2Crypto: 0.20.2
            Mako: Not Installed
          PyYAML: 3.10
           PyZMQ: 14.3.1
          Python: 2.6.6 (r266:84292, Jul 23 2015, 15:22:56)
            RAET: Not Installed
         Tornado: 4.2.1
             ZMQ: 3.2.5
            cffi: Not Installed
        cherrypy: Not Installed
        dateutil: 2.4.2
           gitdb: Not Installed
       gitpython: Not Installed
           ioflo: Not Installed
         libnacl: Not Installed
    msgpack-pure: Not Installed
  msgpack-python: 0.4.6
    mysql-python: Not Installed
       pycparser: 2.14
        pycrypto: 2.6.1
          pygit2: Not Installed
    python-gnupg: Not Installed
           smmap: Not Installed
         timelib: Not Installed

System Versions:
            dist: centos 6.7 Final
         machine: x86_64
         release: 2.6.32-431.el6.x86_64
          system: CentOS 6.7 Final

[root@ans01 salt]# rpm -qa |grep requests
python-requests-2.6.0-3.el6.noarch

[root@ans01 salt]# python -c 'import azure.common; print azure.common.__version__'
1.0.0
[root@ans01 salt]#
rallytime commented 8 years ago

@jliu70 Yes, we don't have salt-cloud working with python azure 1.0.0 quite yet. Their API changed significantly and we're working on adapting salt-cloud's azure driver to work with 1.0 in #27330.

mfussenegger commented 8 years ago

Would it make sense to document this under e.g. https://docs.saltstack.com/en/2015.5/topics/cloud/azure.html until the driver is compatible?

Workaround of using pip install azure==0.10.2 is easy enough, but it is not obvious that it is just a versioning issue.

rallytime commented 8 years ago

@mfussenegger Of course! Good call. I'll do that now.

rallytime commented 8 years ago

@mfussenegger I've added a note about this to the docs. Thanks again for the suggestion. Please feel free to add to it, if you'd like.

rallytime commented 8 years ago

Ok, PR #28571 was just submitted to the develop branch to address using azure 1.0.0. Are any of you in a position to give that a try? We'd love some feedback on if this is working or not.

Also, since this issue is a duplicate of #27330, I am going to close this issue in favor of the previously filed issue. That way we can keep track of everything in one place. If you find any issues with the azure driver in the develop branch, please let us know on #27330. Thanks everyone!

ablinkinz commented 8 years ago

@rallytime I think the message in the docs about the version of the azure provider is no longer valid. in 2016.3.1 I had to install azure 1.0.2 in order to get any azure commands to work. This is the first version I have been able to use above 0.11.1. -Stephan

rallytime commented 8 years ago

@ablinkinz Thanks for letting me know. The msazure.py docs list the correct 1.0.2 version dep, but not the Getting Started with Azure docs. I have fixed this in #34746.

ablinkinz commented 8 years ago

Awesome!

Thanks:

On Jul 18, 2016, at 1:29 PM, Nicole Thomas notifications@github.com wrote:

@ablinkinz Thanks for letting me know. The msazure.py docs list the correct 1.0.2 version dep, but not the Getting Started with Azure docs. I have fixed this in #34746.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.