search

conjure-up / spells

conjure-up spells registry
79 stars 37 forks source link

hulk smashing when fragment adds machines to a bundle with no machines #182

Closed kwmonroe closed 6 years ago

kwmonroe commented 6 years ago

I tried to make the graylog fragment put graylog on machine 0 with a given constraint:

https://github.com/conjure-up/spells/commit/bfd94cf3bd3a3d76e77bc9807b662489504613db

services:
...
   graylog:
     charm: cs:xenial/graylog-15
     num_units: 1
+    to:
+      - "0"
...
+machines:
+  "0":
+    series: xenial
+    constraints: mem=3G

Taking advantage of the graylog addon alias, I run conjure-up like this:

conjure-up --spells-dir </spells/with/updated/graylog> --nosync graylog

This merges my graylog fragment into the following bundle (from ~/.cache/conjure-up/canonical-kubernetes-deployed-20180418.1504.yaml):

applications:
  apache2:
    charm: cs:xenial/apache2-24
    expose: true
    num_units: 1
    options:
      enable_modules: headers proxy_html proxy_http
  easyrsa:
    annotations:
      gui-x: '450'
      gui-y: '550'
    charm: cs:~containers/easyrsa-39
    constraints: root-disk=8G
    num_units: 1
  elasticsearch:
    charm: cs:xenial/elasticsearch-26
    constraints: mem=3G root-disk=16G
    num_units: 1
  etcd:
    annotations:
      gui-x: '800'
      gui-y: '550'
    charm: cs:~containers/etcd-77
    constraints: root-disk=8G
    num_units: 3
  filebeat:
    charm: cs:xenial/filebeat-14
    options:
      kube_logs: true
      logpath: /var/log/*.log
  flannel:
    annotations:
      gui-x: '450'
      gui-y: '750'
    charm: cs:~containers/flannel-52
  graylog:
    charm: cs:xenial/graylog-15
    constraints: mem=7G
    num_units: 1
    to:
    - '0'
  kubeapi-load-balancer:
    annotations:
      gui-x: '450'
      gui-y: '250'
    charm: cs:~containers/kubeapi-load-balancer-57
    constraints: root-disk=8G
    expose: true
    num_units: 1
  kubernetes-master:
    annotations:
      gui-x: '800'
      gui-y: '850'
    charm: cs:~containers/kubernetes-master-102
    constraints: cores=2 mem=4G root-disk=16G
    num_units: 1
    options:
      channel: 1.10/stable
  kubernetes-worker:
    annotations:
      gui-x: '100'
      gui-y: '850'
    charm: cs:~containers/kubernetes-worker-114
    constraints: cores=4 mem=4G root-disk=16G
    expose: true
    num_units: 3
    options:
      channel: 1.10/stable
  mongodb:
    charm: cs:xenial/mongodb-47
    num_units: 1
description: A nine-machine Kubernetes cluster, appropriate for production. Includes
  a three-machine etcd cluster and three Kubernetes worker nodes.
machines:
  '0':
    constraints: mem=3G
    series: xenial
relations:
- - kubernetes-master:kube-api-endpoint
  - kubeapi-load-balancer:apiserver
- - kubernetes-master:loadbalancer
  - kubeapi-load-balancer:loadbalancer
- - kubernetes-master:kube-control
  - kubernetes-worker:kube-control
- - kubernetes-master:certificates
  - easyrsa:client
- - etcd:certificates
  - easyrsa:client
- - kubernetes-master:etcd
  - etcd:db
- - kubernetes-worker:certificates
  - easyrsa:client
- - kubernetes-worker:kube-api-endpoint
  - kubeapi-load-balancer:website
- - kubeapi-load-balancer:certificates
  - easyrsa:client
- - flannel:etcd
  - etcd:db
- - flannel:cni
  - kubernetes-master:cni
- - flannel:cni
  - kubernetes-worker:cni
- - apache2:reverseproxy
  - graylog:website
- - graylog:elasticsearch
  - elasticsearch:client
- - graylog:mongodb
  - mongodb:database
- - filebeat:beats-host
  - kubernetes-master:juju-info
- - filebeat:beats-host
  - kubernetes-worker:juju-info
- - filebeat:logstash
  - graylog:beats
series: xenial

That seems fine, but notice that only graylog has a to: placement directive. However, when the deployment finishes, something afoul is revealed:

$ juju status
...
Unit                      Workload     Agent       Machine  Public address  Ports  Message
apache2/0*                maintenance  executing   0        54.175.104.33          (install) installing charm software
easyrsa/0                 waiting      allocating  1        54.87.218.44           waiting for machine
elasticsearch/0*          maintenance  executing   2        107.20.128.32          (install) installing charm software
etcd/0                    maintenance  executing   3        52.87.172.3            (install) installing charm software
etcd/1*                   maintenance  executing   4        34.239.104.1           (install) installing charm software
etcd/2                    maintenance  executing   5        52.207.229.225         (install) installing charm software
graylog/0*                maintenance  executing   0        54.175.104.33          (install) installing charm software
kubeapi-load-balancer/0*  maintenance  executing   6        54.236.30.175          (install) installing charm software
kubernetes-master/0*      unknown      lost        7        18.232.120.161         agent lost, see 'juju show-status-log kubernetes-master/0'
  filebeat/0*             waiting      allocating           18.232.120.161         agent initializing
  flannel/0*              blocked      executing            18.232.120.161         (install) Waiting for etcd relation.
kubernetes-worker/0       error        lost        8        35.171.203.75          hook failed: "install"
kubernetes-worker/1       active       executing   9        54.175.146.151         (install) Container runtime available.
kubernetes-worker/2*      unknown      lost        10       34.227.29.231          agent lost, see 'juju show-status-log kubernetes-worker/2'
  filebeat/1              waiting      allocating           34.227.29.231          agent initializing
  flannel/1               maintenance  executing            34.227.29.231          (install) installing charm software
mongodb/0*                maintenance  executing   11       52.91.115.222          (install) installing charm software
...

Why is apache2/0 hulk smashed beside graylog/0 on machine 0?

kwmonroe commented 6 years ago

Seems this probably isn't a problem with conjure-up, but rather juju itself. When I juju deploy ~/.cache/conjure-up/canonical-kubernetes-deployed-20180418.1504.yaml, it looks like things are gonna go great, with each app on a new machine and graylog on machine 0:

...
- add unit apache2/0 to new machine 1
- add unit easyrsa/0 to new machine 2
- add unit elasticsearch/0 to new machine 3
- add unit etcd/0 to new machine 4
- add unit etcd/1 to new machine 5
- add unit etcd/2 to new machine 6
- add unit graylog/0 to new machine 0
- add unit kubeapi-load-balancer/0 to new machine 7
- add unit kubernetes-master/0 to new machine 8
- add unit kubernetes-worker/0 to new machine 9
- add unit kubernetes-worker/1 to new machine 10
- add unit kubernetes-worker/2 to new machine 11
- add unit mongodb/0 to new machine 12
Deploy of bundle completed.

However, a juju status on this deployment shows the same thing -- graylog and apache2 are smashed on machine 0:

juju status
...
apache2/0*                maintenance  executing   0        34.238.153.179         (install) installing charm software
easyrsa/0*                active       executing   1        54.89.174.66           (config-changed) Certificate Authority ready.
elasticsearch/0*          maintenance  executing   2        52.203.131.75          (config-changed) installing charm software
etcd/0                    maintenance  executing   3        54.174.144.161         (install) installing charm software
etcd/1                    maintenance  executing   4        54.89.152.120          (install) installing charm software
etcd/2*                   maintenance  executing   5        34.207.94.127          (install) installing charm software
graylog/0*                maintenance  executing   0        34.238.153.179         (install) installing charm software
kubeapi-load-balancer/0*  maintenance  executing   6        174.129.54.106         (install) installing charm software
kubernetes-master/0*      maintenance  executing   7        34.231.247.252         (install) Installing kubectl snap
kubernetes-worker/0       waiting      idle        8        34.234.84.209          Waiting for cluster DNS.
  filebeat/1              maintenance  executing            34.234.84.209          (install) installing charm software
  flannel/1               waiting      allocating           34.234.84.209          agent initializing
kubernetes-worker/1       maintenance  executing   9        34.201.27.75           (install) Installing kube-proxy snap
kubernetes-worker/2*      waiting      idle        10       34.207.89.170          Waiting for cluster DNS.
  filebeat/0*             maintenance  executing            34.207.89.170          (install) Installing filebeat
  flannel/0*              maintenance  executing            34.207.89.170          (install) installing charm software
mongodb/0*                maintenance  executing   11       54.86.240.162          (install) installing charm software
kwmonroe commented 6 years ago

FYI, filed a bug against core with a simpler test case:

https://bugs.launchpad.net/juju/+bug/1765436