error deploy to azure

[duping812]

Expected behavior:

Install streisand service on azure linux vm, locate at west us 2.

Actual Behavior:

TASK [azure-security-group : Create Azure resource group] **** fatal: [localhost]: FAILED! => {"changed": false, "msg": "Do you have azure>=2.0.0 installed? Try pip install 'azure>=2.0.0' --upgrade- No module named azure.mgmt.network.models"}

Steps to Reproduce:

1. I followed the instruction for deploy streisand to azure. Encounter many errors, I tried to solve some of them. First the list of packages in the Prerequisites is not enough. Here are the additional I installed. sudo apt-get update && sudo apt-get install -y libssl-dev libffi-dev sudo pip install azure==2.0.0rc6 sudo pip install 'azure>=2.0.0' --upgrade

After this I can see a new resource group named "streisand-streisand" created, and some successful actions in the log. Right now I am stuck at the azure.mgmt.network, see error message above. Any suggestion would be appreciated.

[ contents of streisand-diagnostics.md here ]

Additional Details:

Log output from Ansible or other relevant services (link to Gist for longer output):

Target Cloud Provider: azure

Operating System of target host: ubuntu 16.04

Operating System of client: windows7

Version of Ansible, using ansible --version : 2.4.2.0

Output from git rev-parse HEAD in your Streisand directory :

9baef3eceeaf94cd823eb376eee00096de2f7b15

[nopdotcom]

Could you try this in a virtualenv? Run

./util/venv-dependencies.sh $HOME/streisand-deps

and follow the directions.

[duping812]

here are the version for the python and azure pakage. python 2.7.12 pip 9.0.1

pip freeze | grep azure azure==2.0.0 azure-batch==3.0.0 azure-cli-core==2.0.22 azure-cli-nspkg==3.0.1 azure-common==1.1.8 azure-datalake-store==0.0.17 azure-graphrbac==0.30.0 azure-keyvault==0.3.7 azure-mgmt==1.0.0 azure-mgmt-authorization==0.30.0 azure-mgmt-batch==4.0.0 azure-mgmt-cdn==0.30.3 azure-mgmt-cognitiveservices==1.0.0 azure-mgmt-commerce==0.30.0rc5 azure-mgmt-compute==1.0.0 azure-mgmt-containerregistry==0.2.1 azure-mgmt-containerservice==1.0.0 azure-mgmt-datalake-analytics==0.1.6 azure-mgmt-datalake-nspkg==2.0.0 azure-mgmt-datalake-store==0.1.6 azure-mgmt-devtestlabs==2.0.0 azure-mgmt-dns==1.0.1 azure-mgmt-documentdb==0.1.3 azure-mgmt-iothub==0.2.2 azure-mgmt-keyvault==0.31.0 azure-mgmt-logic==2.1.0 azure-mgmt-monitor==0.2.1 azure-mgmt-network==1.0.0 azure-mgmt-notificationhubs==0.30.0rc5 azure-mgmt-nspkg==2.0.0 azure-mgmt-powerbiembedded==0.30.0rc5 azure-mgmt-rdbms==0.1.0 azure-mgmt-redis==4.1.1 azure-mgmt-resource==1.1.0 azure-mgmt-scheduler==1.1.3 azure-mgmt-sql==0.5.3 azure-mgmt-storage==1.0.0 azure-mgmt-trafficmanager==0.30.0 azure-mgmt-web==0.32.0 azure-nspkg==2.0.0 azure-servicebus==0.21.1 azure-servicefabric==5.6.130 azure-servicemanagement-legacy==0.20.6 azure-storage==0.34.3 msrestazure==0.4.18

[duping812]

@nopdotcom thanks for the reply. I set up the virtualenv as you suggested, and activated it to run streisand. It pops another error message TASK [azure-security-group : Create Azure resource group] ** fatal: [localhost]: FAILED! => {"changed": false, "msg": "Do you have msrestazure installed? Try pip install msrestazure- No module named msrest.serialization"}

I installed msrestazure and rerun the setup. It backs to the same error message as shown in my original post. tried pip install 'azure>=2.0.0' --upgrade in the virtualenv did not help.

[nopdotcom]

Thank you for trying that. It was a good try.

Could you try my new branch? https://github.com/nopdotcom/streisand.git

git checkout venv-pip2-plus-diagnostics

[duping812]

i clone your git and checkout the branch, the run stuck to same error, with or without virtualenv. I guess this branch will generate some diagnostic outputs right?

[nopdotcom]

Sadly, no. Here's something to try, though.

Look at the first line of type -p ansible. It will give a Python version. For me, in /usr/local/bin/ansible I have

#!/usr/local/opt/python/bin/python2.7

Run that executable path with the Ansible path, and the additional argument -v:

/usr/local/opt/python/bin/python2.7 -v /usr/local/bin/ansible

This will print where all the modules are loaded.

[duping812]

type -p ansible return same as yours, /usr/local/bin/ansible but python path slightly different. I have/usr/bin/python the version is 2.7.12 ran the last command prints lots message. the output is attached. output.txt

Most noticeable is this error.

Some modules do not make sense in Ad-Hoc (include, meta, etc) ERROR! Missing target hosts

[alimakki]

@duping812,

I ran into the same issue you faced when using the virtuan-env

Could you re-activate your virtual env, run the following pip install ansible[azure] and run the streisand script?

[duping812]

@alimakki I activated virtualenv and ran your command. It did install some additional packages. But when i ran the streisand script, it returned same error as in the original post.

[alimakki]

Interesting...

I was curious, what is the operating system of the machine you are using to run the streisand script?

[duping812]

its ubuntu 16.04 vm on azure.

[alimakki]

Hi @duping812.

So a couple of things happened yesterday, first of which is explains why the virtuan-env wasn't working for you.

In the commit you've been using, the venv requirements installed ansible, followed by ansibe[azure]. This seemingly broke the dependencies required for Azure provisioning. The latest master now has that fixed; creating a fresh virtuan-env based on the updated code would be the right course of action.

Onto the second point: there seemingly has been a change in Azure's network APIs that now breaks Ansible tasks when performing actions for security groups. You can follow the issue here if you like. In practice, this means that provisioning towards Azure is not an option at the moment; you may wish to consider another cloud provider in the meantime until upstream pushes a fix.

[duping812]

Hi @alimakki . Thanks for your reply. I cloned the latest master from nopdotcom, the virtualenv set up failed. ./util/venv-dependencies.sh $HOME/streisand-deps

Collecting pycurl (from -r requirements.txt (line 22)) Downloading pycurl-7.43.0.1.tar.gz (195kB) 100% |████████████████████████████████| 204kB 4.5MB/s Complete output from command python setup.py egg_info: Traceback (most recent call last): File "", line 1, in File "/tmp/pip-build-1_4CA2/pycurl/setup.py", line 841, in ext = get_extension(sys.argv, split_extension_source=split_extension_source) File "/tmp/pip-build-1_4CA2/pycurl/setup.py", line 508, in get_extension ext_config = ExtensionConfiguration(argv) File "/tmp/pip-build-1_4CA2/pycurl/setup.py", line 72, in init self.configure() File "/tmp/pip-build-1_4CA2/pycurl/setup.py", line 108, in configure_unix raise ConfigurationError(msg) main.ConfigurationError: Could not run curl-config: [Errno 2] No such file or directory

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-1_4CA2/pycurl/

[alimakki]

I think you may need to run sudo apt-get install libssl-dev libcurl4-openssl-dev before the venv setup.

[duping812]

Great. The venv setup went through. After activate venv, i ran the streisand script. This time it completed more tasks than my original post. azure-security-group executed 4 tasks, now stopped at "Open all of the necessary ports across every service in the Azure security group"

TASK [azure-security-group : Create Azure resource group] ***** ok: [localhost]

TASK [azure-security-group : Create Azure virtual network] **** ok: [localhost]

TASK [azure-security-group : Create Azure subnet] ***** ok: [localhost]

TASK [azure-security-group : Create Azure public ip] ** ok: [localhost]

TASK [azure-security-group : Open all of the necessary ports across every service in the Azure security group] *** An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: init() got multiple values for keyword argument 'id' fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"/tmp/ansible_EInNDK/ansible_module_azure_rm_securitygroup.py\", line 716, in \n main()\n File \"/tmp/ansible_EInNDK/ansible_module_azure_rm_securitygroup.py\", line 712, in main\n AzureRMSecurityGroup()\n File \"/tmp/ansible_EInNDK/ansible_module_azure_rm_securitygroup.py\", line 542, in init\n supports_check_mode=True)\n File \"/tmp/ansible_EInNDK/ansible_modlib.zip/ansible/module_utils/azure_rm_common.py\", line 285, in init\n File \"/tmp/ansible_EInNDK/ansible_module_azure_rm_securitygroup.py\", line 666, in exec_module\n self.results['state'] = self.create_or_update(results)\n File \"/tmp/ansible_EInNDK/ansible_module_azure_rm_securitygroup.py\", line 685, in create_or_update\n parameters.security_rules.append(create_rule_instance(rule))\n File \"/tmp/ansible_EInNDK/ansible_module_azure_rm_securitygroup.py\", line 452, in create_rule_instance\n etag=rule.get('etag', None)\nTypeError: init() got multiple values for keyword argument 'id'\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1} to retry, use: --limit @/home/duping812/nopdotcom/streisand/playbooks/azure.retry

[alimakki]

Yep, unfortunately Microsoft changed their Azure APIs which broke Ansible (mentioned in my previous comment). You can see the bug here: https://github.com/ansible/ansible/issues/33657

[duping812]

Got it. Thanks for the help, @alimakki . I think I will try aws or google cloud instead.

[alimakki]

No problem.