search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.16k stars 5.48k forks source link

salt-cloud cannot provision to azure #28150

Closed dtoubelis closed 8 years ago

dtoubelis commented 9 years ago 
root@scus-saltmaster:~# salt-cloud --versions-report
Salt Version:
            Salt: 2015.8.1

Dependency Versions:
 Apache Libcloud: 0.15.1
          Jinja2: 2.7.2
        M2Crypto: Not Installed
            Mako: 0.9.1
          PyYAML: 3.10
           PyZMQ: 14.4.0
          Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
            RAET: Not Installed
         Tornado: 4.2.1
             ZMQ: 4.0.4
            cffi: Not Installed
        cherrypy: Not Installed
        dateutil: 1.5
           gitdb: 0.5.4
       gitpython: 0.3.2 RC1
           ioflo: Not Installed
         libnacl: Not Installed
    msgpack-pure: Not Installed
  msgpack-python: 0.3.0
    mysql-python: 1.2.3
       pycparser: Not Installed
        pycrypto: 2.6.1
          pygit2: Not Installed
    python-gnupg: Not Installed
           smmap: 0.8.2
         timelib: Not Installed

System Versions:
            dist: Ubuntu 14.04 trusty
         machine: x86_64
         release: 3.19.0-30-generic
          system: Ubuntu 14.04 trusty

When provisioning node I get this:

root@scus-saltmaster:~# salt-cloud -p azure-scus-prod-01-api prod01-api-0001 -l debug
[DEBUG   ] Reading configuration from /etc/salt/cloud
[DEBUG   ] Including configuration from '/etc/salt/cloud.conf.d/20_cache.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.conf.d/20_cache.conf
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/mijem.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/mijem.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: scus-saltmaster.scus-saltmaster.j4.internal.cloudapp.net
[DEBUG   ] Missing configuration file: /etc/salt/cloud.providers
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/azure.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/azure.conf
[DEBUG   ] Missing configuration file: /etc/salt/cloud.profiles
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/azure-servers.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/azure-servers.conf
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/mijem-servers.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/mijem-servers.conf
[DEBUG   ] Configuration file path: /etc/salt/cloud
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] salt-cloud starting
[DEBUG   ] Could not LazyLoad parallels.avail_sizes
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy
[DEBUG   ] Could not LazyLoad saltify.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.avail_images
[DEBUG   ] Could not LazyLoad saltify.avail_locations
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Could not LazyLoad azure.optimize_providers
[DEBUG   ] The 'azure' cloud driver is unable to be optimized.
[DEBUG   ] Could not LazyLoad parallels.avail_sizes
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy
[DEBUG   ] Could not LazyLoad saltify.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.avail_images
[DEBUG   ] Could not LazyLoad saltify.avail_locations
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Failed to execute 'azure.list_nodes()' while querying for running nodes: 'name'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 2266, in run_parallel_map_providers_query
    cloud.clouds[data['fun']]()
  File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/msazure.py", line 226, in list_nodes
    ret[node][prop] = nodes[node][prop]
KeyError: 'name'
[DEBUG   ] Generating minion keys for 'prod01-api-0001'
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event - data = {'profile': 'azure-scus-prod-01-api', 'event': 'starting create', '_stamp': '2015-10-20T19:18:44.203266', 'name': 'prod01-api-0001', 'provider': 'azure:azure'}
[INFO    ] Creating Cloud VM prod01-api-0001
[DEBUG   ] vm_kwargs: {'system_config': <azure.servicemanagement.LinuxConfigurationSet object at 0x7f10ea34a450>, 'deployment_slot': 'production', 'role_size': 'Standard_D2', 'deployment_name': 'prod01-api-0001', 'service_name': 'prod01-api-0001', 'role_name': 'prod01-api-0001', 'network_config': <azure.servicemanagement.ConfigurationSet object at 0x7f10ea33ad10>, 'os_virtual_hard_disk': <azure.servicemanagement.OSVirtualHardDisk object at 0x7f10ea34a650>, 'label': 'prod01-api-0001'}
[DEBUG   ] MasterEvent PUB socket URI: ipc:///var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: ipc:///var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event - data = {'_stamp': '2015-10-20T19:18:44.257005', 'service_kwargs': {'service_name': 'prod01-api-0001', 'label': 'prod01-api-0001', 'location': 'South Central US', 'description': 'prod01-api-0001'}, 'event': 'requesting instance', 'vm_kwargs': {'deployment_slot': 'production', 'role_size': 'Standard_D2', 'deployment_name': 'prod01-api-0001', 'service_name': 'prod01-api-0001', 'label': 'prod01-api-0001', 'role_name': 'prod01-api-0001'}}
[DEBUG   ] vm_kwargs: {'system_config': <azure.servicemanagement.LinuxConfigurationSet object at 0x7f10ea34a450>, 'deployment_slot': 'production', 'role_size': 'Standard_D2', 'deployment_name': 'prod01-api-0001', 'service_name': 'prod01-api-0001', 'role_name': 'prod01-api-0001', 'network_config': <azure.servicemanagement.ConfigurationSet object at 0x7f10ea33ad10>, 'os_virtual_hard_disk': <azure.servicemanagement.OSVirtualHardDisk object at 0x7f10ea34a650>, 'label': 'prod01-api-0001'}
[DEBUG   ] Request ID for machine: b76b15d744332dcba7c62a70cde2e558
[DEBUG   ] Waiting for asynchronous operation to complete
[DEBUG   ] Attempting function <function wait_for_hostname at 0x7f10ea34c5f0>
[DEBUG   ] Caught exception in wait_for_fun: 'name'
[DEBUG   ] Retrying function <function wait_for_hostname at 0x7f10ea34c5f0> on  (try 1)
[DEBUG   ] Caught exception in wait_for_fun: 'name'
[DEBUG   ] Retrying function <function wait_for_hostname at 0x7f10ea34c5f0> on  (try 2)
[DEBUG   ] Caught exception in wait_for_fun: 'name'
[DEBUG   ] Retrying function <function wait_for_hostname at 0x7f10ea34c5f0> on  (try 3)
[DEBUG   ] Caught exception in wait_for_fun: 'name'
...
[DEBUG   ] Retrying function <function wait_for_hostname at 0x7fb5349686e0> on  (try 167)
[ERROR   ] Function timed out: 900
[ERROR   ] Failed to get a value for the hostname.
Error: There was a profile error: Failed to deploy VM

Even though the command times out, the node is created in azure and I can ssh into it but it is not provisioned, not even salt-minion is installed so obviously salt-bootstrap did not run. However, if I try list running instances I get this:

root@scus-saltmaster:~# salt-cloud -f list_nodes azure
[INFO    ] salt-cloud starting
[ERROR   ] There was an error running the function: 'name'
root@scus-saltmaster:~# salt-cloud -f list_nodes azure -l debug
[DEBUG   ] Reading configuration from /etc/salt/cloud
[DEBUG   ] Including configuration from '/etc/salt/cloud.conf.d/20_cache.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.conf.d/20_cache.conf
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/mijem.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/mijem.conf
[DEBUG   ] Including configuration from '/etc/salt/master.d/reactor.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/reactor.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: scus-saltmaster.scus-saltmaster.j4.internal.cloudapp.net
[DEBUG   ] Missing configuration file: /etc/salt/cloud.providers
[DEBUG   ] Including configuration from '/etc/salt/cloud.providers.d/azure.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers.d/azure.conf
[DEBUG   ] Missing configuration file: /etc/salt/cloud.profiles
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/azure-servers.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/azure-servers.conf
[DEBUG   ] Including configuration from '/etc/salt/cloud.profiles.d/mijem-servers.conf'
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles.d/mijem-servers.conf
[DEBUG   ] Configuration file path: /etc/salt/cloud
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] salt-cloud starting
[DEBUG   ] Could not LazyLoad parallels.avail_sizes
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy
[DEBUG   ] Could not LazyLoad saltify.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.avail_images
[DEBUG   ] Could not LazyLoad saltify.avail_locations
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Trying to execute 'azure.list_nodes' with the following kwargs: {}
[ERROR   ] There was an error running the function: 'name'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/cloud/cli.py", line 266, in run
    self.function_provider, self.function_name, kwargs
  File "/usr/lib/python2.7/dist-packages/salt/cloud/__init__.py", line 1517, in do_function
    driver: self.clouds[fun](call='function')
  File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/msazure.py", line 226, in list_nodes
    ret[node][prop] = nodes[node][prop]
KeyError: 'name'
root@scus-saltmaster:~#

I'm painfully aware about limitations that Azure imposes on instance names and all instance names I create are between 3-15 characters long, do not contain dots or other illegal characters and always begin with letter.

Also, trying to delete the instance with salt-cloud also fails with a similar error but unfortunately I do not have trace.

Oh BTW, I'm using azure SDK 0.11.1 since version 1.0.2 does not work with my version of salt.

rallytime commented 9 years ago

Thanks!

FYI @techhat

rallytime commented 9 years ago

@dtoubelis This is a regression. I can confirm this issue with a salt-cloud -f list_nodes azure and can reproduce this stacktrace with 2015.8.1, but just switching back to 2015.8.0 without any other changes works fine.

rallytime commented 9 years ago

@dtoubelis Are you in a position to try the HEAD of 2015.8? I can't reproduce the failure there.

rallytime commented 8 years ago

Since I haven't heard back here, I am going to close this. This is a regression with 2015.8.1 but is fixed at the HEAD of the 2015.8 branch and will be fixed on the upcoming 2015.8.2 branch. If this issue persists when 2015.8.2 is released, let us know and we'll be happy to reopen. Thanks!

dtoubelis commented 8 years ago

Sounds good. Thanks.