search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.01k stars 5.47k forks source link

salt-cloud fails during fixed IPs creation and provisioning GCE instances #56823

Open alrf opened 4 years ago

alrf commented 4 years ago

Description salt-cloud 2019.2.3, gce provider. salt-cloud fails during fixed IPs creation and provisioning GCE instances.

Setup

/etc/salt/cloud.profiles.d/gce.conf:
vpn-profile:
  image: debian-9
  ssh_username: gceuser
  ssh_keyfile: /root/.ssh/id_rsa_gce
  size: f1-micro
  network: default
  subnetwork: default
  tags: '["allow-all"]'
  metadata: '{"sshKeys": "MYKEY"}'
  use_persistent_disk: True
  delete_boot_pd: True
  ssh_interface: public_ips
  ip_forwarding: False
  deploy: True
  make_master: False
  provider: MYPROVIDER
/etc/salt/cloud.maps.d/vpn.conf:
vpn-profile:
  - vpn1:
      location: europe-west4-a
      external_ip: "vpn1"
  - vpn2:
      location: europe-west4-b
      external_ip: "vpn2"
  - vpn3:
      location: europe-west4-c
      external_ip: "vpn3"

According to documentation https://docs.saltstack.com/en/latest/topics/cloud/gce.html#external-ip "vpnX" static (fixed) IPs should be created:

Optionally, pass the name of a GCE address to use a fixed IP address. If the address does not already exist, it will be created.

Steps to Reproduce the behavior Run setup:

/etc/salt/cloud.providers.d# salt-cloud -m /etc/salt/cloud.maps.d/vpn.conf -P
The following virtual machines are set to be created:
  vpn2
  vpn1
  vpn3

Proceed? [N/y] y
... proceeding
[ERROR   ] Caught Exception, terminating workers
TRACE: can't serialize <GCERegion id="1340" name="europe-west4", status="UP">
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 511, in __create_orget_address
    addy = conn.ex_get_address(name, region)
  File "/usr/lib/python3/dist-packages/libcloud/compute/drivers/gce.py", line 6556, in ex_get_address
    response = self.connection.request(request, method='GET').object
  File "/usr/lib/python3/dist-packages/libcloud/compute/drivers/gce.py", line 120, in request
    response = super(GCEConnection, self).request(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/libcloud/common/google.py", line 806, in request
    *args, **kwargs)
  File "/usr/lib/python3/dist-packages/libcloud/common/base.py", line 871, in request
    response = responseCls(**kwargs)
  File "/usr/lib/python3/dist-packages/libcloud/common/base.py", line 182, in __init__
    self.object = self.parse_body()
  File "/usr/lib/python3/dist-packages/libcloud/common/google.py", line 278, in parse_body
    raise ResourceNotFoundError(message, self.status, code)
libcloud.common.google.ResourceNotFoundError: {'domain': 'global', 'reason': 'notFound', 'message': "The resource 'projects/blah/regions/europe-west4/addresses/vpn2' was not found"}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/cloud.py", line 1991, in fire_event
    event.fire_event(msg, tag)
  File "/usr/lib/python3/dist-packages/salt/utils/event.py", line 708, in fire_event
    'Dict object expected, not \'{0}\'.'.format(data)
ValueError: Dict object expected, not 'create address'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/cloud/__init__.py", line 67, in _call
    ret = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/cloud/__init__.py", line 2294, in create_multiprocessing
    local_master=parallel_data['local_master']
  File "/usr/lib/python3/dist-packages/salt/cloud/__init__.py", line 1253, in create
    output = self.clouds[func](vm_)
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 2579, in create
    node_info = request_instance(vm_)
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 2485, in request_instance
    external_ip = __create_orget_address(conn, external_ip, region)
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 517, in __create_orget_address
    new_addy = create_address(addr_kwargs, "function")
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 1263, in create_address
    transport=__opts__['transport']
  File "/usr/lib/python3/dist-packages/salt/utils/cloud.py", line 1998, in fire_event
    event.fire_event(args, tag)
  File "/usr/lib/python3/dist-packages/salt/utils/event.py", line 729, in fire_event
    dump_data = self.serial.dumps(data, use_bin_type=True)
  File "/usr/lib/python3/dist-packages/salt/payload.py", line 240, in dumps
    return msgpack.dumps(msg, default=ext_type_encoder, use_bin_type=use_bin_type)
  File "/usr/lib/python3/dist-packages/msgpack/__init__.py", line 47, in packb
    return Packer(**kwargs).pack(o)
  File "msgpack/_packer.pyx", line 231, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:3661)
  File "msgpack/_packer.pyx", line 233, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:3503)
  File "msgpack/_packer.pyx", line 192, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:2657)
  File "msgpack/_packer.pyx", line 228, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3382)
TypeError: can't serialize <GCERegion id="1340" name="europe-west4", status="UP">

Error: There was a query error: Exception caught
Caught Exception, terminating workers
TRACE: can't serialize <GCERegion id="1340" name="europe-west4", status="UP">
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 511, in __create_orget_address
    addy = conn.ex_get_address(name, region)
  File "/usr/lib/python3/dist-packages/libcloud/compute/drivers/gce.py", line 6556, in ex_get_address
    response = self.connection.request(request, method='GET').object
  File "/usr/lib/python3/dist-packages/libcloud/compute/drivers/gce.py", line 120, in request
    response = super(GCEConnection, self).request(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/libcloud/common/google.py", line 806, in request
    *args, **kwargs)
  File "/usr/lib/python3/dist-packages/libcloud/common/base.py", line 871, in request
    response = responseCls(**kwargs)
  File "/usr/lib/python3/dist-packages/libcloud/common/base.py", line 182, in __init__
    self.object = self.parse_body()
  File "/usr/lib/python3/dist-packages/libcloud/common/google.py", line 278, in parse_body
    raise ResourceNotFoundError(message, self.status, code)
libcloud.common.google.ResourceNotFoundError: {'domain': 'global', 'reason': 'notFound', 'message': "The resource 'projects/blah/regions/europe-west4/addresses/vpn2' was not found"}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/cloud.py", line 1991, in fire_event
    event.fire_event(msg, tag)
  File "/usr/lib/python3/dist-packages/salt/utils/event.py", line 708, in fire_event
    'Dict object expected, not \'{0}\'.'.format(data)
ValueError: Dict object expected, not 'create address'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/cloud/__init__.py", line 67, in _call
    ret = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/cloud/__init__.py", line 2294, in create_multiprocessing
    local_master=parallel_data['local_master']
  File "/usr/lib/python3/dist-packages/salt/cloud/__init__.py", line 1253, in create
    output = self.clouds[func](vm_)
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 2579, in create
    node_info = request_instance(vm_)
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 2485, in request_instance
    external_ip = __create_orget_address(conn, external_ip, region)
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 517, in __create_orget_address
    new_addy = create_address(addr_kwargs, "function")
  File "/usr/lib/python3/dist-packages/salt/cloud/clouds/gce.py", line 1263, in create_address
    transport=__opts__['transport']
  File "/usr/lib/python3/dist-packages/salt/utils/cloud.py", line 1998, in fire_event
    event.fire_event(args, tag)
  File "/usr/lib/python3/dist-packages/salt/utils/event.py", line 729, in fire_event
    dump_data = self.serial.dumps(data, use_bin_type=True)
  File "/usr/lib/python3/dist-packages/salt/payload.py", line 240, in dumps
    return msgpack.dumps(msg, default=ext_type_encoder, use_bin_type=use_bin_type)
  File "/usr/lib/python3/dist-packages/msgpack/__init__.py", line 47, in packb
    return Packer(**kwargs).pack(o)
  File "msgpack/_packer.pyx", line 231, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:3661)
  File "msgpack/_packer.pyx", line 233, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:3503)
  File "msgpack/_packer.pyx", line 192, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:2657)
  File "msgpack/_packer.pyx", line 228, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:3382)
TypeError: can't serialize <GCERegion id="1340" name="europe-west4", status="UP">

So, neither fixed IPs nor GCE instances were created.

Expected behavior "vpnX" fixed IPs and GCE instances should be created, fixed IPs should be mapped to GCE instances, salt-minion and configuration are applied.

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) ``` Salt Version: Salt: 2019.2.3 Dependency Versions: cffi: Not Installed cherrypy: 3.5.0 dateutil: 2.5.3 docker-py: Not Installed gitdb: 2.0.0 gitpython: 2.1.1 ioflo: Not Installed Jinja2: 2.9.4 libgit2: Not Installed libnacl: Not Installed M2Crypto: Not Installed Mako: Not Installed msgpack-pure: Not Installed msgpack-python: 0.4.8 mysql-python: Not Installed pycparser: Not Installed pycrypto: 2.6.1 pycryptodome: Not Installed pygit2: Not Installed Python: 3.5.3 (default, Sep 27 2018, 17:25:39) python-gnupg: 0.3.9 PyYAML: 3.12 PyZMQ: 16.0.2 RAET: Not Installed smmap: 2.0.1 timelib: Not Installed Tornado: 4.4.3 ZMQ: 4.2.1 System Versions: dist: debian 9.12 locale: UTF-8 machine: x86_64 release: 4.9.0-12-amd64 system: Linux version: debian 9.12 ```
twangboy commented 4 years ago

It's saying that it can find vpn2. Is that defined in google cloud? Can you try defining it there first and designating it in the profile?

alrf commented 4 years ago

According to documentation https://docs.saltstack.com/en/latest/topics/cloud/gce.html#external-ip "vpnX" static (fixed) IPs should be created:

Optionally, pass the name of a GCE address to use a fixed IP address. If the address does not already exist, it will be created.

Please read the issue description first. Also, it's saying TypeError: can't serialize <GCERegion id="1340" name="europe-west4", status="UP"> which is definitely serialization issue and related to code.

Another one issue which you already answered, IPs were pre-created here: salt-cloud fails during provisioning GCE instances with pre-created fixed IPs https://github.com/saltstack/salt/issues/56824

sagetherage commented 3 years ago

The Core Team will not be able to get this in this release cycle and moving this back into planning.