salt.cloud.CloudClient method create() not working for VMware driver

[anandnevase]

I am trying to run simple python code to Provision VM using salt.cloud.CloudClient create() API (Create the named VMs, without using a profile) using vmware driver, but unable to execute.

Salt-Cloud Version : salt-cloud 2015.8.3 (Beryllium) Base OS : Centos 6

Sample Code:

import os, json, yaml, collections
from salt.cloud import CloudClient
from salt.exceptions import SaltCloudSystemExit

if __name__ == '__main__':
        client = CloudClient(path='/etc/salt/cloud')
        spec_str='{"provider":"vcenter01","clonefrom":"Win2008R2","num_cpus":4,"memory":"6GB","datastore":"Test-cntr0","cluster":"Test-Cluster","folder":"Test VMs","names":["test_vm_2008"],"devices":{"disk":{"Hard disk 2":{"size":15},"Hard disk 3":{"size":12}},"scsi":{"SCSI controller 1":{"type":"lsilogic"},"SCSI controller 2":{"type":"lsilogic_sas","bus_sharing":"virtual"},"SCSI controller 3":{"type":"paravirtual"}}}}'
        spec=json.loads(spec_str)
        vm_names = spec.get("names")
        ret = client.create(provider='vcenter01', names=vm_names, kwargs=spec)
        print "vm-status: ", ret

Here, I am getting empty json without any exception.

[jfindlay]

@anandxoriant, thanks for the report. Can you post your relevant cloud provider and profile configs with the private info removed? Can you verify that other operations with the provider work as expected using salt-cloud?

[anandnevase]

@jfindlay, following are my cloud.provider & profile configs:

/etc/salt/cloud.providers

vcenter01:
  driver: vmware
  user: '<vcenter-user>'
  password: '<vcenter-pwd>'
  url: '<vcenter-ip>'
  protocol: 'https'
  port: 443
  esxi_host_user: '<esxi-user>'
  esxi_host_password: '<esxi-pwd>'
  esxi_host_ssl_thumbprint: <ssl-thumbprint>
  minion:
    master: <master-ip>
  win_installer: /root/Salt-Minion-2015.8.3-AMD64-Setup.exe
  win_username: <win-admin-user>
  win_password: <win-admin-password>
  smb_port: 445

/etc/salt/cloud.profiles:

vm_prof:
  provider: vcenter01
  clonefrom: Win2008R2
  num_cpus: 4
  memory: 8GB
  datastore: Test-cntr0
  cluster: Test-Cluster
  folder: Test-VMs
  domain: test.com
  devices:
    disk:
      Hard disk 1:
        size: 5
      Hard disk 2:
        size: 2
    scsi:
      SCSI controller 1:
        type: lsilogic
      SCSI controller 2:
        type: lsilogic_sas
        bus_sharing: virtual
      SCSI controller 3:
        type: paravirtual

I can able to provision VM using salt-cloud cli command as well as salt.cloud.CloudClient profile() api. But not able to create VM from salt.cloud.CloudClient create() method

[jfindlay]

@anandxoriant, thanks for the extra info.

@nmadhok

[angystardust]

Maybe I'm wrong but I think that the create function is locked in Beryllium (2015.08) as you can read from this comment https://github.com/saltstack/salt/pull/27324#issuecomment-145947171

It seems that the function was unlocked in the (to be released) Boron release (2016.03) but I haven't tested it yet. https://docs.saltstack.com/en/develop/topics/cloud/vmware.html#creating-a-vm

[funkknight]

I think I got this to work when trying to use salt 'saltmaster-dev01*' cloud.create vcenter salttest.domain.net $lots_o_options....

I would get a return with no output and the debug log of the minion file would show:

2016-04-01 20:49:00,366 [salt.cloud       ][ERROR   ][27556] Failed to create VM salttest.domain.net. Configuration value None needs to be set
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/cloud/__init__.py", line 1274, in create
    output = self.clouds[func](vm_)
  File "/usr/lib/python2.7/site-packages/salt/cloud/clouds/vmware.py", line 2010, in create
    vm_=vm_) is False:
  File "/usr/lib/python2.7/site-packages/salt/config.py", line 2664, in is_profile_configured
    profile_key = opts['providers'][alias][driver]['profiles'][profile_name]
KeyError: None

What I determined was that .create() doesn't set a profile but the vmware.create() function calls config.is_profile_configured(...) which expects one.

I got around it by changing the if statement on line 2007 in salt/cloud/clouds/vmware.py from:

   2007         if config.is_profile_configured(__opts__,
   2008                                     __active_provider_name__ or 'vmware',
   2009                                     vm_['profile'],
   2010                                     vm_=vm_) is False:
   2011             return False

to be

   2007         if vm_['profile'] is not None:
   2008             if config.is_profile_configured(__opts__,
   2009                                         __active_provider_name__ or 'vmware',
   2010                                         vm_['profile'],
   2011                                         vm_=vm_) is False:
   2012                 return False

After this my 'salt ... cloud.create ...' command worked. Doesn't work from within a state however. It presents a different error there which i need to debug.

I am running salt 2015.8.8.2 (Beryllium) HTH

[nmadhok]

@funkknight salt 2015.8.2 or 2015.8.8? Also, what's the error you get from within a state? The change you made on line 2007 will not make sense for users who don't have a profile configured at all since it will skip checking profile configuration altogether.

@anandxoriant Apply the patch from PR #32309 and see if that fixes the issue for you.

[funkknight]

@nmadhok, I'm running 2015.8.8.2, see --versions below.

The patch you show is from a slightly different code base as in my install the file is salt/config.py where as your patch is against salt/config/__init__.py.

I still tried applying the changes in that patch, and removing my tweak. It fails before it hits the changes. The error is the same as in my comment above. Essentially profile_name as passed into salt.config.is_profile_configured can't be None.

I did a quick look to see how other cloud drivers handle the profile issue in def create and they seem to only use the is_profile_configured check if the profile exists:

[root@saltmaster-dev01 clouds]# grep is_profile_configured *.py
aliyun.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
cloudstack.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
digital_ocean.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
ec2.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
gce.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
gogrid.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
joyent.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
libcloud_aws.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
linode.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
msazure.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
nova.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
opennebula.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
openstack.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
parallels.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
proxmox.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
qingcloud.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
rackspace.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
scaleway.py:        if server_['profile'] and config.is_profile_configured(__opts__,
softlayer_hw.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
softlayer.py:        if vm_['profile'] and config.is_profile_configured(__opts__,
vmware.py:            if config.is_profile_configured(__opts__,
vsphere.py:        if vm_['profile'] and config.is_profile_configured(__opts__,

[root@saltmaster-dev01 clouds]# grep is_profile_configured *.py | grep -v vm_
scaleway.py:        if server_['profile'] and config.is_profile_configured(__opts__,
vmware.py:            if config.is_profile_configured(__opts__,

Skipping the profile check if profile is none seems to be inline with the documentation on cloud.create

I just tried the state again and I am getting the same error as the salt cloud.create command. I probably looked at the wrong log file when looking for the state error previously.

[root@saltmaster-dev01 ~]# salt --versions
Salt Version:
           Salt: 2015.8.8.2

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: Not Installed
           Mako: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.7.0
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: 0.8.6
       cherrypy: 3.2.2
       dateutil: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: 0.21.0
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
         pygit2: 0.21.4
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: centos 7.2.1511 Core
        machine: x86_64
        release: 3.10.0-327.10.1.el7.x86_64
         system: CentOS Linux 7.2.1511 Core

[rallytime]

@funkknight Does #32344 help resolve that final piece of this bug for you?

[funkknight]

It does work. Thanks!

[anandnevase]

@rallytime Still python API salt.cloud.CloudClient.create() not working.

[rallytime]

@anandnevase Can you be a little more specific about what isn't working? Have you applied the two patches referenced above?