search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.1k stars 5.47k forks source link

type error when creating instances from profile with salt-cloud in [2016.11.0] #38060

Closed moremo closed 7 years ago

moremo commented 7 years ago

Description of Issue/Question

Receive type error when creating instances from profile with salt-cloud in 2016,11.0, does not occur with same setup in 2016.3.4.

Setup

cloud.providers

srs-aws-config:
  id: use-instance-role-credentials
  key: use-instance-role-credentials
  keyname: SRV-VPC
  private_key: /root/SRV-VPC-priv.pem
  ssh_interface: private_ips
  location: eu-west-1
  driver: ec2
  del_root_vol_on_destroy: True
  del_all_vols_on_destroy: True
  rename_on_destroy: True

cloud.profiles

int2_app:
  provider: srs-aws-config
  image: ami-55084526
  size: t2.micro
  network_interfaces:
    - DeviceIndex: 0
      PrivateIpAddresses:
        - Primary: True
      AssociatePublicIpAddress: False
      SubnetId: subnet-sor4xmg
      SecurityGroupId:
        - sg-3246syxd
  volumes:
    - { size: 10, device: /dev/sdf, type: gp2 }
    - { size: 20, device: /dev/sdg, type: gp2 }
  grains:
    node_type: app
  iam_profile: 'arn:aws:iam::1234567891011:instance-profile/srs-DEV-ROLE'
  sync_after_install: grains
  win_installer: /root/Salt-Minion-2016.3.2-AMD64-Setup.exe
  win_username: Administrator
  win_password: auto
  minion:
    master: 10.19.4.36

Steps to Reproduce Issue

running salt-cloud -p int2_app int2-app-ms03 -l debug

gives 

salt-cloud -p int2_app int2-app-ms03 -l debug
[DEBUG   ] Reading configuration from /etc/salt/cloud
[DEBUG   ] Reading configuration from /etc/salt/master
[DEBUG   ] Including configuration from '/etc/salt/master.d/salt-api.conf'
[DEBUG   ] Reading configuration from /etc/salt/master.d/salt-api.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: ip-10-19-4-35.eu-west-1.compute.internal
[DEBUG   ] Reading configuration from /etc/salt/cloud.providers
[DEBUG   ] Reading configuration from /etc/salt/cloud.profiles
[DEBUG   ] Configuration file path: /etc/salt/cloud
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[INFO    ] salt-cloud starting
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Could not LazyLoad parallels.avail_sizes: 'parallels' __virtual__ returned False
[DEBUG   ] LazyLoaded parallels.avail_locations
[DEBUG   ] LazyLoaded proxmox.avail_sizes
[DEBUG   ] Could not LazyLoad saltify.destroy: 'saltify.destroy' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_sizes: 'saltify.avail_sizes' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_images: 'saltify.avail_images' is not available.
[DEBUG   ] Could not LazyLoad saltify.avail_locations: 'saltify.avail_locations' is not available.
[DEBUG   ] LazyLoaded rackspace.reboot
[DEBUG   ] LazyLoaded openstack.list_locations
[DEBUG   ] LazyLoaded rackspace.list_locations
[DEBUG   ] Using AWS endpoint: ec2.eu-west-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.eu-west-1.amazonaws.com/?Action=DescribeInstances&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] LazyLoaded cloud.cache_node_list
[DEBUG   ] Generating minion keys for 'int2-app-ms03'
[DEBUG   ] LazyLoaded cloud.fire_event
[DEBUG   ] MasterEvent PUB socket URI: /var/run/salt/master/master_event_pub.ipc
[DEBUG   ] MasterEvent PULL socket URI: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Initializing new IPCClient for path: /var/run/salt/master/master_event_pull.ipc
[DEBUG   ] Sending event: tag = salt/cloud/int2-app-ms03/creating; data = {'profile': 'int2_app', 'event': 'starting create', '_stamp': '2016-12-02T19:13:38.364970', 'name': 'int2-
app-ms03', 'provider': 'srs-aws-config:ec2'}
[INFO    ] Creating Cloud VM int2-app-ms03 in eu-west-1
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] compile template: :string:
[DEBUG   ] Jinja search path: ['/var/cache/salt/cloud/files/base']

[DEBUG   ] Create network interface: {'SubnetId': 'subnet-sor4xmg', 'DeviceIndex': 0, 'SecurityGroupId': ['sg-3246syxd'], 'PrivateIpAddresses': [{'Primary': True}], 'AssociatePubl
icIpAddress': False}
[DEBUG   ] Using AWS endpoint: ec2.eu-west-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.eu-west-1.amazonaws.com/?Action=DescribeSubnets&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[INFO    ] Attempting to look up root device name for image id ami-55084526 on VM int2-app-ms03
[DEBUG   ] Using AWS endpoint: ec2.eu-west-1.amazonaws.com
[DEBUG   ] AWS Request: https://ec2.eu-west-1.amazonaws.com/?Action=DescribeImages&ImageId.1=ami-55084526&Version=2014-10-01
[DEBUG   ] AWS Response Status Code: 200
[DEBUG   ] EC2 Response: '[{'virtualizationType': 'hvm', 'imageOwnerAlias': 'amazon', 'name': 'Windows_Server-2012-R2_RTM-English-64Bit-Base-2016.10.12', 'imageOwnerId': '801119661
308', 'hypervisor': 'xen', 'isPublic': 'true', 'sriovNetSupport': 'simple', 'imageId': 'ami-55084526', 'imageState': 'available', 'platform': 'windows', 'blockDeviceMapping': {'ite
m': [{'deviceName': '/dev/sda1', 'ebs': {'snapshotId': 'snap-cd52a13a', 'deleteOnTermination': 'true', 'volumeSize': '30', 'volumeType': 'gp2', 'encrypted': 'false'}}, {'deviceName
': 'xvdca', 'virtualName': 'ephemeral0'}, {'deviceName': 'xvdcb', 'virtualName': 'ephemeral1'}, {'deviceName': 'xvdcc', 'virtualName': 'ephemeral2'}, {'deviceName': 'xvdcd', 'virtu
alName': 'ephemeral3'}, {'deviceName': 'xvdce', 'virtualName': 'ephemeral4'}, {'deviceName': 'xvdcf', 'virtualName': 'ephemeral5'}, {'deviceName': 'xvdcg', 'virtualName': 'ephemera
l6'}, {'deviceName': 'xvdch', 'virtualName': 'ephemeral7'}, {'deviceName': 'xvdci', 'virtualName': 'ephemeral8'}, {'deviceName': 'xvdcj', 'virtualName': 'ephemeral9'}, {'deviceName
': 'xvdck', 'virtualName': 'ephemeral10'}, {'deviceName': 'xvdcl', 'virtualName': 'ephemeral11'}, {'deviceName': 'xvdcm', 'virtualName': 'ephemeral12'}, {'deviceName': 'xvdcn', 'vi
rtualName': 'ephemeral13'}, {'deviceName': 'xvdco', 'virtualName': 'ephemeral14'}, {'deviceName': 'xvdcp', 'virtualName': 'ephemeral15'}, {'deviceName': 'xvdcq', 'virtualName': 'ep
hemeral16'}, {'deviceName': 'xvdcr', 'virtualName': 'ephemeral17'}, {'deviceName': 'xvdcs', 'virtualName': 'ephemeral18'}, {'deviceName': 'xvdct', 'virtualName': 'ephemeral19'}, {'
deviceName': 'xvdcu', 'virtualName': 'ephemeral20'}, {'deviceName': 'xvdcv', 'virtualName': 'ephemeral21'}, {'deviceName': 'xvdcw', 'virtualName': 'ephemeral22'}, {'deviceName': 'x
vdcx', 'virtualName': 'ephemeral23'}, {'deviceName': 'xvdcy', 'virtualName': 'ephemeral24'}, {'deviceName': 'xvdcz', 'virtualName': 'ephemeral25'}]}, 'architecture': 'x86_64', 'ima
geLocation': 'amazon/Windows_Server-2012-R2_RTM-English-64Bit-Base-2016.10.12', 'rootDeviceType': 'ebs', 'creationDate': '2016-10-12T19:33:53.000Z', 'rootDeviceName': '/dev/sda1',
'imageType': 'machine', 'description': 'Microsoft Windows Server 2012 R2 RTM 64-bit Locale English AMI provided by Amazon'}]'
[INFO    ] Found root device name: /dev/sda1
[ERROR   ] There was a profile error: 'NoneType' object has no attribute '__getitem__'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/cloud/cli.py", line 284, in run
    self.config.get('names')
  File "/usr/lib/python2.7/site-packages/salt/cloud/__init__.py", line 1457, in run_profile
    ret[name] = self.create(vm_)
  File "/usr/lib/python2.7/site-packages/salt/cloud/__init__.py", line 1287, in create
    output = self.clouds[func](vm_)
  File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/ec2.py", line 2509, in create
    data, vm_ = request_instance(vm_, location)
  File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/ec2.py", line 1890, in request_instance
    if 'Ebs.VolumeType' not in ex_blockdevicemappings[dev_index]:
TypeError: 'NoneType' object has no attribute '__getitem__'

Versions Report

Dependency Versions:
           cffi: Not Installed
       cherrypy: 3.2.2
       dateutil: 1.5
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: 0.21.1
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.5 (default, Oct 11 2015, 17:47:16)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: redhat 7.2 Maipo
        machine: x86_64
        release: 3.10.0-327.el7.x86_64
         system: Linux
        version: Red Hat Enterprise Linux Server 7.2 Maipo
Ch3LL commented 7 years ago

@moremo can you give this fix a try: https://github.com/saltstack/salt/pull/37967 ?

I believe this is a duplicate of https://github.com/saltstack/salt/issues/37966 but the issue never stated what the error was, but looking at your stack trace i believe the fix in that PR will fix the issue. Please let me know.

moremo commented 7 years ago

Sorry, looked around for a matching issue, but didn't connect with that one. That appears to resolve the issue.

Ch3LL commented 7 years ago

Thanks for getting back. Yeah i'm glad you posted the error so it will be a little easier for users to search for if they run into the issue as well. Thanks :)