Tectonic Baremetal - Behind Proxy

reignblack commented 7 years ago

Issue Report Template

Tectonic Version

1.5.5.2

Environment

What hardware/cloud provider/hypervisor is being used with Tectonic? Bare Metal

Expected Behavior

Installation to complete

Actual Behavior

Installation looks to have stopped on hung on etcd.member.service

Reproduction Steps

VLAN Subet 10.96.104.0/24 1 Provisioner Node – 10.96.104.10 – Running DHCP, Matchbox and Tectonic Installer via gui 1 Controller – 10.96.104.20 received from DHCP, DNS entry server-9001.example.com 1 Node – 10.96.104.22 received from DHCP, DNS entry server-9002.example.com

Followed baremetal documentation and setup matchbox and pxe boot, only difference is I didnt use the coreos dns/dhcp and used my own
Run Tectonic Installer 1.5.5.2

--Cluster Name – kube-a --Controller DNS – kube-a.example.com – DNS record to 104.20 --Tectonic DNS – tectonic-a.example.com – DNS record to 104.22 ---Certificates – Self Signed --Matchbox Server – matchbox.example.com:8080/8081 – DNS record to 10.96.104.10 --Pod Range - 172.16.0.0/16 --Service Range – 172.17.0.0/24 --Etcd – Launch/install etcd for me

Power on Cluster --Controller – Gets IP from DHCP, loads PXE boot and installs OS --Node – Gets IP from DHCP, loads PXE boot and installs OS --Both servers boot and then reboot, I leave them for a while, no visual update on installer

I can log into both the controller and node and by running a journalctl –f can see errors relating to the etcd-member.service, so ran journalctl -u etcd-member.service and get the following error:

etcd-wrapper[18385]: + exec /usr/bin/rkt run --uuid-file-save=/var/lib/coreos/etcd-member-wrapper.uuid (have cut this off as its quite long and made the post messy)
etcd-wrapper[18385]: run: discovery failed
etcd-member.service: Main process exited, code=exited, status=254/n/a
rkt [18404]: stop: unable to resolve UUID from file: open /var/lib/coreos/etcd-member-wrapper.uuid: no such file or directory

So as it seemed like it was trying to download it, i realized this wouldn't be possible as it is behind a corporate proxy. I then ran a fetch command from rkt, and it would fail.

I googled and found the following https://gist.github.com/alekssaul/59a87b5be86f77cc4638083250d90568

Which made some sense to me, but it was older version of tectonic and I wasn't getting etc running. So I proceeded anyway and added the proxy information into the yaml file, deleted the controller and node disks, and rebooted again from pxe again.

Nodes came up fine, but still the same error, but if run a rkt fetch it is now able to download. If followed through with the other steps on that post using the updated version numbers, the installer would report kubernetes is now running, but still etcd component is not, but if i fetch etcd the tectonic installer and restart the service the installer will report its running.

I can get to the next stage, but again it fails and needs bootkube, so have downloaded that on the controller, but it seems im getting flannel errors now. At the moment im more interested in resolving the proxy part so it can download the necessary files, instead of me manually trying to get this to work.

Other Information

Linux/coreos is new to me, so im not the best at troubleshooting and I could have this all wrong, but happy to provide any more information.

Feature Request

Help on getting it running behind a proxy. Where/how do i set the the right proxy settings? Its seems there are multiple places this needs to be set? Would you need to set the System wide environment variables as mentioned on the coreos documentation?

Environment

What hardware/cloud provider/hypervisor is being used with Tectonic? Bare Metal

Desired Feature

Add configuration page for http/https/noproxy on tectonic installer

Other Information

reignblack commented 7 years ago

Update: I added the following to both the controller and worker yaml files.

 - path: /etc/systemd/system.conf.d/10-default-env.conf
      filesystem: root
      mode: 0644
      contents:
        inline: |
          [Manager]
          DefaultEnvironment=HTTP_PROXY=http://myproxy1:8080

- path: /etc/profile.env
      filesystem: root
      mode: 0644
      contents:
        inline: |
          export HTTP_PROXY=http://myproxy1:8080

The installation now completes successfully and i now get the console login screen. Although I can fix this manually, it still would be great if this was added to the installer.

mfburnett commented 7 years ago

Hey @reignblack, thanks for documenting your workaround. I'll bring this to the team and keep you updated.

rushins commented 7 years ago

Hello i ran into the same issue . i did the same thing but nothing working ... ?

Update: I added the following to both the controller and worker yaml files.

path: /etc/systemd/system.conf.d/10-default-env.conf filesystem: root mode: 0644 contents: inline: | [Manager] DefaultEnvironment=HTTP_PROXY=http://myproxy1:8080
- path: /etc/profile.env filesystem: root mode: 0644 contents: inline: | export HTTP_PROXY=http://myproxy1:8080

reignblack commented 7 years ago

Sorry I perhaps could of been more clear in my answer.

You will also need to add the proxy values for docker.service and early-docker.service

Please see https://gist.github.com/alekssaul/59a87b5be86f77cc4638083250d90568

edevil commented 6 years ago

Do we still need to add the env config or is there some tectonic argument to enable a proxy?

knweiss commented 6 years ago

@edevil It's still a problem. FWIW: https://github.com/coreos/tectonic-installer/issues/38 was just reopened.

coreos / tectonic-forum