[v3-beta4] installer fails

[thoraxe]

TASK: [openshift_master | Create master config] ******************************* 
failed: [ose3-master.example.com] => {"changed": true, "cmd": ["/usr/bin/openshift", "start", "master", "--write-config", "--config=/etc/openshift/master.yaml", "--portal-net=172.30.17.0/24", "--master=https://ose3-master.example.com:8443", "--public-master=https://ose3-master.example.com:8443", "--listen=https://0.0.0.0:8443", "--images=openshift3_beta/ose-${component}:${version}"], "delta": "0:00:00.735496", "end": "2015-05-05 19:45:47.363989", "rc": 1, "start": "2015-05-05 19:45:46.628493", "warnings": []}
stderr: Error: flag needs an argument: --write-config
Run 'openshift help' for usage.

FATAL: all hosts have already failed -- aborting

Not sure if I have the latest vars and such. Going to double check.

[thoraxe]

The syntax of --config and --write-config has changed.

It looks like --write-config expects a directory as an argument (eg: /etc/openshift/master) and --config actually doesn't do the same thing anymore.

Now it appears that --config expects a file as input that will provide all of the configuration options.

--write-config appears to only want to write out master-config.yaml and I don't see anywhere to change it. Specifying a full filename like --write-config=/etc/openshift/master/default.yaml creates a folder called default.yaml, which is not what we want.

I'm messing around with Ansible now but I ended up getting past this issue and am now having certificate issues.

[thoraxe]

So I tweaked some vars:

openshift_master_config_dir: /etc/openshift/master
openshift_master_config: /etc/openshift/master/master-config.yaml
openshift_master_ca_dir: "{{ openshift_cert_dir }}/ca"
openshift_master_ca_cert: "{{ openshift_master_ca_dir }}/cert.crt"
openshift_master_ca_key: "{{ openshift_master_ca_dir }}/key.key"

And then modified the "Create master config" section:

- name: Create master config
  command: >
    /usr/bin/openshift start master --write-config={{ openshift_master_config_dir }}
    --portal-net={{ openshift.master.portal_net }}
    --master={{ openshift.master.api_url }}
    --public-master={{ openshift.master.public_api_url }}
    --listen={{ 'https' if openshift.master.api_use_ssl else 'http' }}://0.0.0.0:{{ openshift.master.api_port }}
    {{ ('--images=' ~ openshift_registry_url) if (openshift_registry_url | default('', true) != '') else '' }}
    {{ ('--nodes=' ~ openshift_node_ips | join(',')) if (openshift_node_ips | default('', true) != '') else '' }}
  args:
    chdir: "{{ openshift_cert_parent_dir }}"
    creates: "{{ openshift_master_config }}"

This results in the following /etc/openshift/master/master-config.yaml:

apiVersion: v1
assetConfig:
  logoutURL: ""
  masterPublicURL: https://192.168.133.2:8443
  publicURL: https://192.168.133.2:8443/console/
  servingInfo:
    bindAddress: 0.0.0.0:8443
    certFile: master.server.crt
    clientCA: ""
    keyFile: master.server.key
corsAllowedOrigins:
- 127.0.0.1
- 192.168.133.2:8443
- localhost
dnsConfig:
  bindAddress: 0.0.0.0:53
etcdClientInfo:
  ca: ca.crt
  certFile: master.etcd-client.crt
  keyFile: master.etcd-client.key
  urls:
  - https://192.168.133.2:4001
etcdConfig:
  address: 192.168.133.2:4001
  peerAddress: 192.168.133.2:7001
  peerServingInfo:
    bindAddress: 0.0.0.0:7001
    certFile: etcd.server.crt
    clientCA: ca.crt
    keyFile: etcd.server.key
  servingInfo:
    bindAddress: 0.0.0.0:4001
    certFile: etcd.server.crt
    clientCA: ca.crt
    keyFile: etcd.server.key
  storageDirectory: /root/openshift-ansible/openshift.local.etcd
etcdStorageConfig:
  kubernetesStorageVersion: v1beta3
  openShiftStorageVersion: v1beta1
imageConfig:
  format: openshift/origin-${component}:${version}
  latest: false
kind: MasterConfig
kubeletClientInfo:
  ca: ca.crt
  certFile: master.kubelet-client.crt
  keyFile: master.kubelet-client.key
  port: 10250
kubernetesMasterConfig:
  masterIP: 192.168.133.2
  schedulerConfigFile: ""
  servicesSubnet: 172.30.0.0/16
  staticNodeNames: []
masterClients:
  deployerKubeConfig: openshift-deployer.kubeconfig
  externalKubernetesKubeConfig: ""
  openshiftLoopbackKubeConfig: openshift-client.kubeconfig
masterPublicURL: https://192.168.133.2:8443
oauthConfig:
  assetPublicURL: https://192.168.133.2:8443/console/
  grantConfig:
    method: auto
  identityProviders:
  - challenge: true
    login: true
    name: anypassword
    provider:
      apiVersion: v1
      kind: AllowAllPasswordIdentityProvider
  masterPublicURL: https://192.168.133.2:8443
  masterURL: https://192.168.133.2:8443
  sessionConfig:
    sessionMaxAgeSeconds: 300
    sessionName: ssn
    sessionSecretsFile: ""
  tokenConfig:
    accessTokenMaxAgeSeconds: 86400
    authorizeTokenMaxAgeSeconds: 300
policyConfig:
  bootstrapPolicyFile: policy.json
  masterAuthorizationNamespace: master
  openshiftSharedResourcesNamespace: openshift
projectRequestConfig:
  projectRequestMessage: ""
  projectRequestTemplate: openshift/project-request
servingInfo:
  bindAddress: 0.0.0.0:8443
  certFile: master.server.crt
  clientCA: ca.crt
  keyFile: master.server.key

So it appears that a bunch of stuff has moved around / been busted in the latest upstream 0.5.0.0 puddle.

[thoraxe]

Certificates are now ending up in /etc/openshift/master:

[root@ose3-master openshift-ansible]# ll /etc/openshift/master
total 148
-rw-r--r--. 1 root root  1115 May  5 20:21 admin.crt
-rw-------. 1 root root  1679 May  5 20:21 admin.key
-rw-------. 1 root root  7176 May  5 20:21 admin.kubeconfig
-rw-r--r--. 1 root root  1066 May  5 20:21 ca.crt
-rw-------. 1 root root  1675 May  5 20:21 ca.key
-rw-r--r--. 1 root root     2 May  5 20:21 ca.serial.txt
-rw-r--r--. 1 root root  2335 May  5 20:21 etcd.server.crt
-rw-------. 1 root root  1679 May  5 20:21 etcd.server.key
-rw-r--r--. 1 root root  2494 May  5 20:21 master-config.yaml
-rw-r--r--. 1 root root  1070 May  5 20:21 master.etcd-client.crt
-rw-------. 1 root root  1679 May  5 20:21 master.etcd-client.key
-rw-r--r--. 1 root root  1070 May  5 20:21 master.kubelet-client.crt
-rw-------. 1 root root  1679 May  5 20:21 master.kubelet-client.key
-rw-r--r--. 1 root root  2335 May  5 20:21 master.server.crt
-rw-------. 1 root root  1675 May  5 20:21 master.server.key
-rw-r--r--. 1 root root  1086 May  5 20:21 openshift-client.crt
-rw-------. 1 root root  1675 May  5 20:21 openshift-client.key
-rw-------. 1 root root  7165 May  5 20:21 openshift-client.kubeconfig
-rw-r--r--. 1 root root  1123 May  5 20:21 openshift-deployer.crt
-rw-------. 1 root root  1675 May  5 20:21 openshift-deployer.key
-rw-------. 1 root root  7223 May  5 20:21 openshift-deployer.kubeconfig
-rw-r--r--. 1 root root  1127 May  5 20:21 openshift-registry.crt
-rw-------. 1 root root  1675 May  5 20:21 openshift-registry.key
-rw-------. 1 root root  7227 May  5 20:21 openshift-registry.kubeconfig
-rw-r--r--. 1 root root  1119 May  5 20:21 openshift-router.crt
-rw-------. 1 root root  1675 May  5 20:21 openshift-router.key
-rw-------. 1 root root  7209 May  5 20:21 openshift-router.kubeconfig
-rw-r--r--. 1 root root 16991 May  5 20:21 policy.json

[thoraxe]

https://github.com/detiber/openshift-ansible/pull/26

[thoraxe]

https://github.com/openshift/origin/issues/2081

I don't think we're going to get traction on this one.

I think it might pay to abandon using the config generators and switch to using a templated YAML file.

Otherwise, if we want to conform to the FHS (https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard) we probably need the certificates to be in /var/lib/openshift/...

[detiber]

@thoraxe, I'm roping in @sdodson since he has been working on the v3-beta4 changes in this PR: https://github.com/openshift/openshift-ansible/pull/198

[thoraxe]

Oh, great. Missed that. At least I learned something about Ansible. I guess just close and reject my pr unless there's something interesting in it. I got to the same issue with etcd and SDN ssl certificates.

[detiber]

Closing since this has been merged into the v3-beta4 branch already.

[nkhare]

I am seeing the issue with v3-beta4 as well. failed: [gprfc077.sbu.lab.eng.bos.redhat.com] => {"changed": true, "cmd": ["/usr/bin/openshift", "start", "master", "--write-config", "--config=/etc/openshift/master.yaml", "--portal-net=172.30.17.0/24", "--master=https://ose3-master.example.com:8443", "--public-master=https://ose3-master.example.com:8443", "--listen=https://0.0.0.0:8443", "--images=openshift3_beta/ose-${component}:${version}"], "delta": "0:00:00.050026", "end": "2015-06-08 05:23:47.655775", "rc": 1, "start": "2015-06-08 05:23:47.605749", "warnings": []} stderr: Error: flag needs an argument: --write-config Run 'openshift help' for usage. stdout: Usage: openshift start master [options]

[sdodson]

@nkhare Please make sure you have the latest code from detiber/v3-beta4 as the error you've got only happened under older code bases.