Deploy k8s cluster fails with ctr: mkdir /opt/pharos/bin: no such file or directory\n"

[raghavanv90]

When I try to deploy the k8s cluster it fails with a below error, Environment: VMs - Tried it on both versions - CentOS Linux release 7.4.1708 (Core) & CentOS Linux release 7.8.2003 (Core) pharos 3.2.1 chpharos 0.6.2

`ERROR: Pharos::PhaseManager::Error : Phase failed on 1 host:

• master: Pharos::ExecError: "exec failed with code 1: configure-cfssl.sh\n + sudo env -i - http_proxy= https_proxy= no_proxy= HTTP_PROXY= HTTPS_PROXY= NO_PROXY= FTP_PROXY= PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin IMAGE=docker.io/jakolehm/cfssl:0.1.1 bash --norc --noprofile -x -s\n + set -e\n + ctr -n pharos image pull docker.io/jakolehm/cfssl:0.1.1\n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 0.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 0.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 0.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 0.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 0.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 0.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 0.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 0.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 0.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.0 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 1.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.0 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 2.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.0 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 3.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.0 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 4.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.0 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 5.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.0 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 6.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.0 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 7.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.0 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 8.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.0 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.1 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.2 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.3 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.4 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.5 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.6 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.7 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.8 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 9.9 s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 10.0s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 10.1s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 10.2s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 10.3s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 10.4s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 10.5s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 10.6s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 10.7s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolving |\e[32m\e[0m--------------------------------------| \n elapsed: 10.8s total: 0.0 B (0.0 B/s) \n \ docker.io/jakolehm/cfssl:0.1.1: resolved \ |\e[32m++++++++++++++++++++++++++++++++++++++\e[0m| \n index-sha256:060f8665eb7a898674ca59bd38366f04ffab7393efae6151534cb0603b2fb078: \ done |\e[32m++++++++++++++++++++++++++++++++++++++\e[0m| \n manifest-sha256:d33d9dfe8ba032f8b1def5055109d8b641d2cfe178681853f72031b5a5a67e8a: done |\e[32m++++++++++++++++++++++++++++++++++++++\e[0m| \n layer-sha256:98ad46cc3fd5ddd614beded63bbac8beb19d0df2577af9b9b3ee2af11a74ad71: \ done |\e[32m++++++++++++++++++++++++++++++++++++++\e[0m| \n config-sha256:19538d3f06f39f850d5d12b7af44ce8fb803d6d84bd8bd34f39f39043fbf9d52: \ done |\e[32m++++++++++++++++++++++++++++++++++++++\e[0m| \n layer-sha256:df8bb6a9cbcb760ed579e83f9dfc232836053c3f099df62822a6231812d1598f: \ done |\e[32m++++++++++++++++++++++++++++++++++++++\e[0m| \n elapsed: 10.9s total: \ 0.0 B (0.0 B/s) \n unpacking linux/amd64 sha256:060f8665eb7a898674ca59bd38366f04ffab7393efae6151534cb0603b2fb078...\n \ done\n + ctr -n pharos install --path /opt/pharos --replace docker.io/jakolehm/cfssl:0.1.1\n \ ctr: mkdir /opt/pharos/bin: no such file or directory\n" `

[root@localhost ~]# cat cluster.yml hosts:

• address: "10.20.XX.XX" user: root ssh_key_path: ~/.ssh/id_rsa role: master
• address: "10.20.XX.XX" user: root ssh_key_path: ~/.ssh/id_rsa role: worker network: provider: weave addon_paths:
• "./pure-addons" addons: ingress-nginx: enabled: true configmap:

  see all supported options: https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/configmap.md

  load-balance: least_conn cert-manager: enabled: true issuer: name: letsencrypt-staging server: https://acme-staging-v02.api.letsencrypt.org/directory email: xxxx.com host-upgrades: enabled: true schedule: "30 6 *" schedule_window: 1h reboot: true

  interval: 7d

  kontena-storage: enabled: true data_dir: /var/lib/kontena-storage storage: use_all_nodes: true pool: replicated: size: 3 dashboard: enabled: true filesystem: enabled: true pool: replicated: size: 3 kontena-backup: enabled: false cloud_credentials: /path/to/aws_credentials aws: bucket: pharos-backups region: eu-central-1 kontena-lens: enabled: true name: 'pharos-cluster' ingress: host: 'mycluster.local' tls: enabled: true email: 'xxxx.com' user_management: enabled: true persistence: enabled: true charts: enabled: true repositories:

  • name: stable url: https://kubernetes-charts.storage.googleapis.com/ helm: enabled: true

[rwunderer]

Same problem here on Debian 9. The strange thing is that neither pharos-cluster nor jakolehm/cfssl have been updated recently and it was working last week.

For the moment the workaround seems to be to manually run

sudo mkdir /opt/pharos

on the master(s)

[Kerwood]

I am experiencing the same issue on 3 newly installed CentOS 7.8 machines.

Below is the pharos up command in debug mode.

...
    10.6.70.11: + ctr -n pharos image pull docker.io/jakolehm/cfssl:0.1.1
    10.6.70.11: docker.io/jakolehm/cfssl:0.1.1: resolving      |--------------------------------------| 

     ## OUTPUT OMITTED ##

    10.6.70.11: docker.io/jakolehm/cfssl:0.1.1:                                                   resolved       |++++++++++++++++++++++++++++++++++++++| 

     ## OUTPUT OMITTED ##

    10.6.70.11: elapsed: 1.6 s                                                                    total:   0.0 B (0.0 B/s)                                         
    10.6.70.11: unpacking linux/amd64 sha256:060f8665eb7a898674ca59bd38366f04ffab7393efae6151534cb0603b2fb078...
    10.6.70.11: done
    10.6.70.11: + ctr -n pharos install --path /opt/pharos --replace docker.io/jakolehm/cfssl:0.1.1
    10.6.70.11: ctr: mkdir /opt/pharos/bin: no such file or directory
    10.6.70.11: ! 1
    [node01] Error: exec failed with code 1: configure-cfssl.sh
...

https://github.com/kontena/pharos-cluster/blob/master/lib/pharos/host/el7/scripts/configure-cfssl.sh

Maybe there's breaking changes to ctr? Maybe the --path option used to create the directory if it did not exist. But doesn't any more.

DESCRIPTION:

ctr is an unsupported debug and administrative client for interacting
with the containerd daemon. Because it is unsupported, the commands,
options, and operations are not guaranteed to be backward compatible or
stable from release to release of the containerd project.

[Kerwood]

I can confirm. ctr from v1.3.0 and up does not create the missing directory, if it does not exist. I created a pull request.

[raghavanv90]

@rwunderer yeah workaround seems to solve the issue However observing other issue after manually creating "sudo mkdir /opt/pharos" - Opened https://github.com/kontena/pharos-cluster/issues/1559, are you facing the same?

[jakolehm]

Fixed in #1556

[jberger]

Following the directions on the site I hit this problem today still. Its seems like your fix has been applied, is there a new iteration of the same bug or have has it not reached the latest release/documentation?

[jberger]

Ok well I guess the problem is that it is fixed in pharos 3.2.2 and current is still 3.2.1 as of installing using chpharos today. Could that get pushed?