Closed roffe closed 6 years ago
I have found the issue with is stupidity from my side as well as imho, poor documentation..
The whole environment is started with PXE boot over DHCP but the tectonics installer requires mac adress & hostname, it uses this hostname to connect to the nodes to provision them, but the hostnames cannot be provisioned before the machines are deployed as they are DHCP assigned.
I guess this is a catch 22:
One cannot provide a complete hostname with IP until the machines are booted and one knows what random IP the DHCP server assigned it & tectonics cannot provision the hosts without fixed hostnames which is gonna be hard when the machines ip's are assigned by DHCP
Which leads to the next question: Am i supposed to run DHCP with fixed leases and fixed hostnames pre tied to ip adresses to get this to work?
I got around it by adding the ip's the nodes got to the hostnames after the innitial bootstrap. after that the cluster creation could continue
hello roffe, i got the same issue but i setup up DNS entries of the IP booted from tectonic installer. can you guide me some hint, i get this error Failed to start Determine the Kubelet Image Version. -- Subject: Unit kubelet-env.service has failed. i see the kubelet-env service is failing with condition failed.
============ sudo systemctl status kubelet-env.service ● kubelet-env.service - Determine the Kubelet Image Version Loaded: loaded (/etc/systemd/system/kubelet-env.service; enabled; vendor preset: enabled) Active: inactive (dead) (Result: exit-code) since Wed 2017-08-16 05:21:22 UTC; 12min ago Condition: start condition failed at Wed 2017-08-16 05:21:25 UTC; 12min ago └─ ConditionPathExists=!/etc/kubernetes/kubelet.env was not met Process: 10542 ExecStartPre=/usr/bin/bash -c docker run --rm -v /etc/kubernetes/kubeconfig:/kubeconfig quay.io/coreos/kube-version:0.1.0 --kubeconfig=/kubeconfig > /etc/kubernetes/kube.ve Process: 10535 ExecStartPre=/bin/mkdir -p /etc/kubernetes (code=exited, status=0/SUCCESS) CPU: 38ms
Aug 16 05:21:11 lvpal3013.pal.sa.corp systemd[1]: kubelet-env.service: Unit entered failed state. Aug 16 05:21:11 lvpal3013.pal.sa.corp systemd[1]: kubelet-env.service: Failed with result 'exit-code'. Aug 16 05:21:22 lvpal3013.pal.sa.corp systemd[1]: kubelet-env.service: Service hold-off time over, scheduling restart. Aug 16 05:21:22 lvpal3013.pal.sa.corp systemd[1]: Stopped Determine the Kubelet Image Version.
=================
do you have any idea how to fix this...
This is happening to me. I put the MAC Addresses into DHCP so the nodes and master would pull the correct IP during bootup. I have verified this. But the cluster will not form. Just getting constant spam about the docker port entered a disabled state.
Issue: The etcd certificates are missing. It seems that there is no /etc/ssl/etcd directory.
<< Solution >>
The tectonic installer cannot reach your newly created nodes. The instaler will ssh into the nodes using the private key of the ssh key you inserted in the tectonic cluster creation wizard. Before begining the setup you will need to add your ssh private key to your ssh agent running on the machine which is running the tectonic installer.
Form other users: "Make sure the SSH key specifyied in the tfvars file is also added to the SSH agent on the machine running terraform. Without this, terraform is not able to SSH copy the assets and start bootkube. Also make sure that the SSH known_hosts file doesn't have old records of the API DNS name (fingerprints will not match Tectonic installer will add the installer machine's public SSH key to all machines in the cluster. The key must be on the installing machine's ssh-agent, and it is used to configure nodes. Check if a key already exists in the ssh-agent using ssh-add -l. If a key must be added to the agent, use ssh-add Path/ToYour/KeyFile. Note that on OSX it may be necessary to re-add keys from your keyring to the agent on each login."*
Steps to take:
On the machine that runs the tectonic installer (in my case I ran the tectonic installer on the matchbox machine) do this:
1) make sure ssh-agent is running: $eval $(ssh-agent -s) 2) check your currentlly attached keys (none by default) $ssh-add -L 3) add your ssh private key that coresponds to the ssh-rsa key you will be using in the installer $ssh-add ~/.ssh/id_rsa
if you don't have any keys you can generate one like this: $ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
Good luck! Alexandru
On Fri, Oct 13, 2017 at 8:20 PM, tiagoaquino notifications@github.com wrote:
I'm having the same problem "/etc/ssl/etcd: stat /etc/ssl/etcd: no such file or directory" instaling on bare metal.
ps: PXE boot, DHCP, DNS, matchbox... everything works fine.
Any solution please?? Thanks
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/coreos/tectonic-installer/issues/1379#issuecomment-336514611, or mute the thread https://github.com/notifications/unsubscribe-auth/AAj9J8soFhwzl2TaDoNDTVBOJOxx1Uo8ks5sr5vNgaJpZM4OWnFS .
Thanks @alexandrufulop !
Solved my problem!!! :)
What keywords did you search in tectonic-installer issues before filing this one?
null_resource.etcd_secrets: Still creating
Is this a BUG REPORT or FEATURE REQUEST?
Choose one: BUG REPORT
Versions
terraform version
): Terraform v0.9.6What happened?
Followed your instructions regarding network, pxe boot dhcp etc. Tectonics fails to create a cluster or even make a etcd node..
PXE boot and everything works. the machines gets installed, rebooted, get their hostnames entered as master & worker. Then the scripts starts failing.
What you expected to happen?
Tectonics magic 15 minutes to kubernetes to kick in
How to reproduce it (as minimally and precisely as possible)?
Follow your guides: https://coreos.com/tectonic/docs/latest/install/bare-metal/ https://coreos.com/matchbox/docs/0.6.1/network-setup.html https://coreos.com/matchbox/docs/latest/deployment.html#coreos
Anything else we need to know?
PXE boot, DHCP, DNS everything works. Terraform manifest is created on server, pushed to machine son power on and they get "installed" after that everything that looks like it's suposed to be the kubernetes bootstrap fails.
Log from tectonics: tectonic-agi-cph.txt
Output from failing master node:
Output form failing controller node install: