search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.09k stars 5.47k forks source link

Kubernetes state report newly created namespace as failed #46063

Open jryberg opened 6 years ago

jryberg commented 6 years ago

Description of Issue/Question

When I create a new namespace using salt.states.kubernetes.namespace_present it's failing even if it's successfully created.

Setup

namespace.sls

Create kubernetes namespace newspace-1:
  kubernetes.namespace_present:
    - name: newspace-1

Kubernetes Python client version 3: https://github.com/saltstack/salt/issues/44701

Steps to Reproduce Issue

Access to Kubernetes cluster running 1.9.2 Create namespace.sls Run the state and this will be the end result

[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Including configuration from '/etc/salt/minion.d/basic.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/basic.conf
[DEBUG   ] Configuration file path: /etc/salt/minion
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Including configuration from '/etc/salt/minion.d/basic.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/basic.conf
[DEBUG   ] Please install 'virt-what' to improve results of the 'virtual' grain.
[DEBUG   ] Connecting to master. Attempt 1 of 1
[DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'bofh', 'tcp://x.x.x.x:4506')
[DEBUG   ] Generated random reconnect delay between '1000ms' and '11000ms' (10353)
[DEBUG   ] Setting zmq_reconnect_ivl to '10353ms'
[DEBUG   ] Setting zmq_reconnect_ivl_max to '11000ms'
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/minion', 'bofh', 'tcp://x.x.x.x:4506', 'clear')
[DEBUG   ] Decrypting the current master AES key
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] Determining pillar cache
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/minion', 'bofh', 'tcp://x.x.x.x:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'bofh', 'tcp://x.x.x.x:4506')
[DEBUG   ] Loaded minion key: /etc/salt/pki/minion/minion.pem
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded state.apply
[DEBUG   ] LazyLoaded saltutil.is_running
[DEBUG   ] LazyLoaded grains.get
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/minion', 'bofh', 'tcp://x.x.x.x:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'bofh', 'tcp://x.x.x.x:4506')
[INFO    ] Loading fresh modules for state activity
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] Could not find file 'salt://kubernetes/tools/create_namespace.sls' in saltenv 'base'
[DEBUG   ] In saltenv 'base', looking at rel_path 'namespace.sls' to resolve 'salt://namespace.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/minion/files/base/namespace.sls' to resolve 'salt://namespace.sls'
[DEBUG   ] compile template: /var/cache/salt/minion/files/base/namespace.sls
[DEBUG   ] Jinja search path: ['/var/cache/salt/minion/files/base']
[DEBUG   ] LazyLoaded pillar.get
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/namespace.sls' using 'jinja' renderer: 0.00538086891174
[DEBUG   ] Rendered data from file: /var/cache/salt/minion/files/base/namespace.sls:
# Create Kubernetes namespace
Create kubernetes namespace newspace-1:
  kubernetes.namespace_present:
    - name: newspace-1

[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Results of YAML rendering:
OrderedDict([('Create kubernetes namespace newspace-1', OrderedDict([('kubernetes.namespace_present', [OrderedDict([('name', 'newspace-1')])])]))])
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/namespace.sls' using 'yaml' renderer: 0.00371098518372
[DEBUG   ] LazyLoaded kubernetes.ping
[DEBUG   ] LazyLoaded kubernetes.namespace_present
[INFO    ] Running state [newspace-1] at time 10:30:56.391504
[INFO    ] Executing state kubernetes.namespace_present for [newspace-1]
[DEBUG   ] response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces \"newspace-1\" not found","reason":"NotFound","details":{"name":"newspace-1","kind":"namespaces"},"code":404}

[DEBUG   ] response body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"newspace-1","selfLink":"/api/v1/namespaces/newspace-1","uid":"78ea97e3-1304-11e8-a19d-0631a2b8350e","resourceVersion":"127463","creationTimestamp":"2018-02-16T10:30:56Z"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}}

[ERROR   ] Made the following changes:
'namespace' changed from 'absent' to '{'status': {'phase': 'Active'}, 'kind': 'Namespace', 'spec': {'finalizers': ['kubernetes']}, 'api_version': 'v1', 'metadata': {'name': 'newspace-1', 'owner_references': None, 'generation': None, 'namespace': None, 'labels': None, 'generate_name': None, 'deletion_timestamp': None, 'cluster_name': None, 'finalizers': None, 'deletion_grace_period_seconds': None, 'initializers': None, 'self_link': '/api/v1/namespaces/newspace-1', 'resource_version': '127463', 'creation_timestamp': datetime.datetime(2018, 2, 16, 10, 30, 56, tzinfo=tzlocal()), 'annotations': None, 'uid': '78ea97e3-1304-11e8-a19d-0631a2b8350e'}}'

[INFO    ] Completed state [newspace-1] at time 10:30:56.427394 duration_in_ms=35.891
[DEBUG   ] File /var/cache/salt/minion/accumulator/140384521860560 does not exist, no need to cleanup.
[DEBUG   ] LazyLoaded config.option
[DEBUG   ] Initializing new AsyncZeroMQReqChannel for ('/etc/salt/pki/minion', 'bofh', 'tcp://x.x.x.x:4506', 'aes')
[DEBUG   ] Initializing new AsyncAuth for ('/etc/salt/pki/minion', 'bofh', 'tcp://x.x.x.x:4506')
[DEBUG   ] LazyLoaded highstate.output
[DEBUG   ] LazyLoaded nested.output
local:
----------
          ID: Create kubernetes namespace newspace-1
    Function: kubernetes.namespace_present
        Name: newspace-1
      Result: False
     Comment:
     Started: 10:30:56.391503
    Duration: 35.891 ms
     Changes:
              ----------
              namespace:
                  ----------
                  new:
                      ----------
                      api_version:
                          v1
                      kind:
                          Namespace
                      metadata:
                          ----------
                          annotations:
                              None
                          cluster_name:
                              None
                          creation_timestamp:
                              ¸20180216T10:30:56.000000
                          deletion_grace_period_seconds:
                              None
                          deletion_timestamp:
                              None
                          finalizers:
                              None
                          generate_name:
                              None
                          generation:
                              None
                          initializers:
                              None
                          labels:
                              None
                          name:
                              newspace-1
                          namespace:
                              None
                          owner_references:
                              None
                          resource_version:
                              127463
                          self_link:
                              /api/v1/namespaces/newspace-1
                          uid:
                              78ea97e3-1304-11e8-a19d-0631a2b8350e
                      spec:
                          ----------
                          finalizers:
                              - kubernetes
                      status:
                          ----------
                          phase:
                              Active
                  old:
                      ----------

Summary for local
------------
Succeeded: 0 (changed=1)
Failed:    1
------------
Total states run:     1
Total run time:  35.891 ms

Maybe the output is parsed wrongly? 'status': {'phase': 'Active'} was the end result

The namespace was created

root@bofh:~# kubectl get namespaces
NAME          STATUS    AGE
newspace-1    Active    17m
default       Active    1d
kube-public   Active    1d
kube-system   Active    1d
root@ip-10-157-0-10:~#

Versions Report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Master

Salt Version:
           Salt: 2017.7.3

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Dec  4 2017, 14:50:18)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-112-generic
         system: Linux
        version: Ubuntu 16.04 xenial

Minion

Salt Version:
           Salt: 2017.7.3

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Dec  4 2017, 14:50:18)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-1049-aws
         system: Linux
        version: Ubuntu 16.04 xenial
Ch3LL commented 6 years ago

ping @flavio looks like you added this here: https://github.com/saltstack/salt/pull/39514

mind taking a look at this?

jodok commented 6 years ago

i can confirm that issue. when the namespace is newly created it is reported as failed (the first couple of lines are also printed red in the shell) ...

myhost.com:
----------
          ID: my_namespace
    Function: kubernetes.namespace_present
        Name: my_ns
      Result: False
     Comment:
     Started: 23:44:42.937843
    Duration: 29.003 ms
     Changes:
              ----------
              namespace:
                  ----------
                  new:
                      ----------
                      api_version:
                          v1
                      kind:
                          Namespace
                      metadata:
                          ----------
                          annotations:
                              None
                          cluster_name:
                              None
                          creation_timestamp:
                              ¸20180227T23:44:42.000000
                          deletion_grace_period_seconds:
                              None
                          deletion_timestamp:
                              None
                          finalizers:
                              None
                          generate_name:
                              None
                          generation:
                              None
                          initializers:
                              None
                          labels:
                              None
                          name:
                              my_ns
                          namespace:
                              None
                          owner_references:
                              None
                          resource_version:
                              836415
                          self_link:
                              /api/v1/namespaces/my_ns
                          uid:
                              2f15dab9-1c18-11e8-865f-000d3a2cdd3b
                      spec:
                          ----------
                          finalizers:
                              - kubernetes
                      status:
                          ----------
                          phase:
                              Active
                  old:
                      ----------
Failed:    1
stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

jryberg commented 5 years ago

Has it been fixed?

stale[bot] commented 5 years ago

Thank you for updating this issue. It is no longer marked as stale.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale[bot] commented 4 years ago

Thank you for updating this issue. It is no longer marked as stale.