search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.09k stars 5.47k forks source link

Profile Error when trying to generate an Azure VM from GCP #47506

Closed justin-w-long closed 6 years ago

justin-w-long commented 6 years ago

Description of Issue/Question

My Salt Master is in GCP and I am trying to get vm generation in Azure to work from a GCP master. I am able to communicate with Azure and it begins the process of spinning up the VM but it fails during the creation with a profile error.

Setup

Provider Configuration:

azure-ubuntu:
  driver: azurearm
  subscription_id: SUBSCRITION_ID

  tenant: TENANT_ID
  client_id: CLIENT_ID
  secret: MY_SECRET

  location: eastus
  resource_group: AzureDemo
  network_resource_group: AzureDemo
  storage_account: MY_STORAGE_ACCOUNT
  storage_key: MY_STORAGE_KEY

  cleanup_disks: True
  cleanup_vhds: True
  cleanup_data_disks: True
  cleanup_interfaces: True

  # Set up the location of the salt master
  #
  minion:
    master: MASTER_IP

Profile Configuration:

azure-ubuntu:
  provider: azure-ubuntu
  image: Canonical|UbuntuServer|18.04-LTS|18.04.201804262
  size: Standard_DS1
  ssh_username: MY_USERNAME
  ssh_password: MY_PASSWORD
  location: eastus

  network: AzureDemo-Network
  public_ip: False

Steps to Reproduce Issue

Run the command: "salt-cloud -l debug -p azure-ubuntu test-demo"

[ERROR ] There was a profile error: init() takes exactly 1 argument (2 given) Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/salt/cloud/cli.py", line 281, in run self.config.get('names') File "/usr/lib/python2.7/dist-packages/salt/cloud/init.py", line 1454, in runprofile ret[name] = self.create(vm) File "/usr/lib/python2.7/dist-packages/salt/cloud/init.py", line 1284, in create output = self.cloudsfunc File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1279, in create 'wait_for_ip_intervalmultiplier', vm, opts, default=1), File "/usr/lib/python2.7/dist-packages/salt/utils/cloud.py", line 2389, in wait_for_ip data = update_callback(*update_args, **update_kwargs) File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1254, in _query_ip_address data = requestinstance(kwargs=vm) File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1052, in request_instance iface_data = createinterface(kwargs=vm) File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 926, in create_interface six.text_type(pub_ip_data.id), # pylint: disable=no-member TypeError: init() takes exactly 1 argument (2 given)

Versions Report

Salt Version: Salt: 2018.3.0

Dependency Versions: cffi: 1.11.5 cherrypy: 3.5.0 dateutil: 2.7.2 docker-py: Not Installed gitdb: 0.6.4 gitpython: 1.0.1 ioflo: Not Installed Jinja2: 2.8 libgit2: Not Installed libnacl: Not Installed M2Crypto: 0.21.1 Mako: 1.0.3 msgpack-pure: Not Installed msgpack-python: 0.4.6 mysql-python: Not Installed pycparser: 2.18 pycrypto: 2.6.1 pycryptodome: Not Installed pygit2: Not Installed Python: 2.7.12 (default, Dec 4 2017, 14:50:18) python-gnupg: Not Installed PyYAML: 3.11 PyZMQ: 15.2.0 RAET: Not Installed smmap: 0.9.0 timelib: Not Installed Tornado: 4.2.1 ZMQ: 4.1.4

System Versions: dist: Ubuntu 16.04 xenial locale: UTF-8 machine: x86_64 release: 4.4.0-72-generic system: Linux version: Ubuntu 16.04 xenial

gtmanfred commented 6 years ago

@nicholasmhughes can you help me with this one?

Thanks, Daniel

nicholasmhughes commented 6 years ago

@justin-w-long can you post the output of pip freeze | egrep 'azure|msrest' on the master?

justin-w-long commented 6 years ago 
azure==3.0.0
azure-batch==4.0.0
azure-batch-extensions==1.0.1
azure-cli==2.0.30
azure-cli-acr==2.0.22
azure-cli-acs==2.0.30
azure-cli-advisor==0.5.0
azure-cli-appservice==0.1.30
azure-cli-backup==1.1.0
azure-cli-batch==3.1.11
azure-cli-batchai==0.1.6
azure-cli-billing==0.1.7
azure-cli-cdn==0.0.13
azure-cli-cloud==2.0.12
azure-cli-cognitiveservices==0.1.11
azure-cli-command-modules-nspkg==2.0.1
azure-cli-configure==2.0.14
azure-cli-consumption==0.2.2
azure-cli-container==0.1.20
azure-cli-core==2.0.30
azure-cli-cosmosdb==0.1.19
azure-cli-dla==0.0.18
azure-cli-dls==0.0.20
azure-cli-eventgrid==0.1.11
azure-cli-eventhubs==0.1.1
azure-cli-extension==0.0.11
azure-cli-feedback==2.1.0
azure-cli-find==0.2.8
azure-cli-interactive==0.3.18
azure-cli-iot==0.1.18
azure-cli-keyvault==2.0.20
azure-cli-lab==0.0.20
azure-cli-monitor==0.1.4
azure-cli-network==2.0.26
azure-cli-nspkg==3.0.2
azure-cli-profile==2.0.21
azure-cli-rdbms==0.2.0
azure-cli-redis==0.2.11
azure-cli-reservations==0.1.1
azure-cli-resource==2.0.26
azure-cli-role==2.0.21
azure-cli-servicebus==0.1.1
azure-cli-servicefabric==0.0.11
azure-cli-sql==2.0.24
azure-cli-storage==2.0.30
azure-cli-vm==2.0.29
azure-common==1.1.8
azure-cosmosdb-nspkg==2.0.2
azure-cosmosdb-table==1.0.2
azure-datalake-store==0.0.19
azure-eventgrid==0.1.0
azure-graphrbac==0.40.0
azure-keyvault==0.3.7
azure-mgmt==2.0.0
azure-mgmt-advisor==1.0.1
azure-mgmt-applicationinsights==0.1.1
azure-mgmt-authorization==0.40.0
azure-mgmt-batch==4.1.0
azure-mgmt-batchai==0.2.0
azure-mgmt-billing==0.1.0
azure-mgmt-cdn==1.0.0
azure-mgmt-cognitiveservices==1.0.0
azure-mgmt-commerce==1.0.1
azure-mgmt-compute==4.0.0rc1
azure-mgmt-consumption==2.0.0
azure-mgmt-containerinstance==0.4.0
azure-mgmt-containerregistry==1.0.1
azure-mgmt-containerservice==3.0.1
azure-mgmt-cosmosdb==0.3.1
azure-mgmt-datafactory==0.4.0
azure-mgmt-datalake-analytics==0.2.0
azure-mgmt-datalake-nspkg==2.0.0
azure-mgmt-datalake-store==0.2.0
azure-mgmt-devtestlabs==2.0.0
azure-mgmt-dns==2.0.0rc1
azure-mgmt-eventgrid==0.4.0
azure-mgmt-eventhub==1.2.0
azure-mgmt-hanaonazure==0.1.0
azure-mgmt-iothub==0.4.0
azure-mgmt-iothubprovisioningservices==0.1.0
azure-mgmt-keyvault==0.40.0
azure-mgmt-loganalytics==0.1.0
azure-mgmt-logic==2.1.0
azure-mgmt-machinelearningcompute==0.4.0
azure-mgmt-managementpartner==0.1.0
azure-mgmt-marketplaceordering==0.1.0
azure-mgmt-media==0.2.0
azure-mgmt-monitor==0.5.0
azure-mgmt-msi==0.1.0
azure-mgmt-network==2.0.0rc1
azure-mgmt-notificationhubs==1.0.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-powerbiembedded==1.0.0
azure-mgmt-rdbms==1.0.0
azure-mgmt-recoveryservices==0.1.0
azure-mgmt-recoveryservicesbackup==0.1.1
azure-mgmt-redis==4.1.0
azure-mgmt-relay==0.1.0
azure-mgmt-reservations==0.1.0
azure-mgmt-resource==1.2.1
azure-mgmt-scheduler==1.1.3
azure-mgmt-search==1.0.0
azure-mgmt-servermanager==1.2.0
azure-mgmt-servicebus==0.4.0
azure-mgmt-servicefabric==0.1.0
azure-mgmt-sql==0.8.5
azure-mgmt-storage==1.5.0
azure-mgmt-subscription==0.1.0
azure-mgmt-trafficmanager==0.40.0
azure-mgmt-web==0.35.0
azure-multiapi-storage==0.2.0
azure-nspkg==2.0.0
azure-servicebus==0.21.1
azure-servicefabric==6.1.2.9
azure-servicemanagement-legacy==0.20.6
azure-storage==0.34.3
azure-storage-blob==1.1.0
azure-storage-common==1.1.0
azure-storage-file==1.1.0
azure-storage-nspkg==3.0.0
azure-storage-queue==1.1.0
You are using pip version 8.1.1, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
msrest==0.4.27
msrestazure==0.4.25
nicholasmhughes commented 6 years ago

@justin-w-long thanks! It looks like you're failing on a section of code that should only execute when public_ip: True... so, that's weird. I tried it with the same options and got past that section of code. I subsequently failed because I didn't provide storage account information in my provider. You might consider using the Azure driver from develop, because it has a few nice updates and most notably includes the ability to use managed disks (the MS recommended option).

Also, I noticed that you're setting public_ip: False, but I don't see deploy: False... which will cause you problems during bootstrap. If you don't need minion software on the host, then it's no problem. Otherwise, the code in develop will allow you to pass a userdata script as a VM Extension to install Salt and point it at your master's public IP.

@gtmanfred , do you know when we are going to see the Azure code from develop hit a stable release? It looks like the code in 2018.3 and develop are diverging...

justin-w-long commented 6 years ago

It looks like it is still generating a public IP address in the Azure Console even though the value is set to False.

nicholasmhughes commented 6 years ago

Yeah... I can't explain that. It's not happening to me. Make sure there aren't any weird characters in your profile definition file. It could cause that parameter to get missed.

In other news, I reproduced a similar TypeError to what you were seeing. It seems that newer versions of the Azure SDK don't like ambiguity in their object models.

This failed:

                    ip_kwargs['public_ip_address'] = PublicIPAddress(
                        six.text_type(pub_ip_data.id),  # pylint: disable=no-member
)

...but adding the explicit id= worked:

                    ip_kwargs['public_ip_address'] = PublicIPAddress(
                        id=six.text_type(pub_ip_data.id),  # pylint: disable=no-member
)

Also, I found some other "interesting" behavior in the newer version of the Azure SDK surrounding strings vs. unicode.

From /usr/lib/python2.7/site-packages/azure/mgmt/resource/resources/resource_management_client.py:

        if not isinstance(subscription_id, str):
            raise TypeError("Parameter 'subscription_id' must be str.")

...that's just sloppy.

The short version is that you might have to downgrade your Azure SDK versions until things "just work".

justin-w-long commented 6 years ago

So I realized I had another profile that I was testing that had the same profile name causing the public IP to still be spun up. After removing that other profile it makes it pass the portion of the code for publicIP, but throws the same error not long after when interacting with disk name and storage information.

[ERROR ] There was a profile error: init() takes exactly 1 argument (2 given) Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/salt/cloud/cli.py", line 281, in run self.config.get('names') File "/usr/lib/python2.7/dist-packages/salt/cloud/init.py", line 1454, in runprofile ret[name] = self.create(vm) File "/usr/lib/python2.7/dist-packages/salt/cloud/init.py", line 1284, in create output = self.cloudsfunc File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1279, in create 'wait_for_ip_intervalmultiplier', vm, opts, default=1), File "/usr/lib/python2.7/dist-packages/salt/utils/cloud.py", line 2389, in wait_for_ip data = update_callback(*update_args, **update_kwargs) File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1254, in _query_ip_address data = requestinstance(kwargs=vm) File "/usr/lib/python2.7/dist-packages/salt/cloud/clouds/azurearm.py", line 1177, in request_instance disk_name, TypeError: init() takes exactly 1 argument (2 given)

nicholasmhughes commented 6 years ago

I think your best best is downgrading these Azure SDK modules:

azure-mgmt-compute==3.1.0rc3
azure-mgmt-network==1.7.1

I had success creating an instance with the versions I listed above and the latest 2018.3 code from Git which includes the logic to ensure subscription_id is a string.

justin-w-long commented 6 years ago

I am now able to get a virtual machine to generate but it fails because it says my subscription ID is not a string. I am on version 2018.3 according to salt --version, do I need to reinstall it or something to get the logic to make sure my subscription_id is a string?

nicholasmhughes commented 6 years ago

It's in the 2018.3 branch, which is where patch releases will get cut. It's pretty safe to pull it from there.

justin-w-long commented 6 years ago

After getting the latest 2018.3 branch code and rolling back those versions of azure-mgmt-compute and network I am able to spin up VM's now. Thank you for all your help!

gtmanfred commented 6 years ago

Thanks for the help @nicholasmhughes

It is greatly appreciated!

Daniel