kn init openstack creates empty directory

[michaelosthege]

Hi there, My colleague @kossikater and I are trying to use KN to init a cluster on OpenStack. Since we are running on Windows workstations (long story), we have created a container that has access to the hosts docker command:

docker run -it -v /var/run/docker.sock:/var/run/docker.sock ubuntu:latest sh -c "apt-get update ; apt-get install docker.io -y ; bash"

Then we proceed with the Getting Started:

docker pull kubenow/provisioners

Here we did not yet understand what the pull is for, because from the instructions we understand that the subsequent commands are also run from the hosts terminal (the container in our case), but not from a container preconfigured by you.

Without sudo because within the container we're root already:

curl -Lo kn https://raw.githubusercontent.com/kubenow/KubeNow/master/bin/kn && chmod +x kn && mv kn /usr/local/bin

At this point the kn command becomes available. So far so good. Next, we downloaded and sourced the OpenStack RC file from our tenacy.

kn init openstack our_deployment

The init command runs through without errors, notifying us about the creation of an SSH keypair, but the directory that is created remains empty! From this point on, obviously we can not proceed.

Did we miss something? Are we supposed to be running all these steps from within a provisioning-container, for example?

[mcapuccini]

Hi! Thanks for reporting. I think Docker runs on a VM on windows, and perhaps creates the directory in the VM, and something there goes wrong... We don't support Windows officially, but @carmat88 and @andersla may manage to help you anyway. I'll be on vacation for three weeks from now. I'll give it a look when I'll be back!

[carmat88]

Howdy @michaelosthege ! Thanks for pointing this out. As @mcapuccini mentioned, honestly we haven't never supported and tested solutions on Windows workstations. Which in turn means : we are unsure why that happens.

It's going to be a busy period now for colleagues are (or about to go) on leave and we have soon an important deadline. Which means that unfortunately we cannot prioritize this. But thanks a lot again for opening the issue!

However, I can reckon that you may have some work to be run and tested. Therefore I was thinking: given that you seem to have an openstack installation (or provider) available, wouldn't be easier if you launch a Linux instance and use that as a VM to run our getting started scenario? (i.e. you ssh into such VM and just work as described in our guidelines). If you could, then I'd bet that dealing with the deploying procedure would be way easier. Furthermore, being supposedly both the VM and cluster on the same network, data transfer and connection should go smoothly.

Would this temporarily workaround be feasible for you guys?

Cheers.

[andersla]

I like @carmat88 Idea about starting a "Deployer" instance in the OpenStack tenancy. I tried the "Docker in Docker" that you did on my Ubuntu machine and it failed the same way with an empty directory as you described. Although I probably won't have time to investigate further at the moment at least I can reproduce the error. On the roadmap is converting our kn-bash script to Windows Power Shell (but that is not due for months). Yet another alternative would be to launch the script from a Ubuntu machine in Windows VirtualBox.

[michaelosthege]

We found the problem with the directory:

Even though we were running the kn commands from within a container, but the docker engine was still the one of the host OS. Thereby the volume bind in the docker_run method of the kn script failed (silently) because it can not bind container B to a folder within container A. (This also explains with the same approach fails on Ubuntu Docker in Docker as @andersla described.)

We modified the kn script to mount the folder from the docker-host/Windows drive instead. Now both, our "deployer" container and the containers created by kn.docker_run bind to the same directory.

We have now run into the next problem: The kn apply fails due to an error within terraform. It looks like the openrc.sh file we downloaded from our (de.NBI) OpenStack tenacy sets the environment variables in a different way than the (outdated) terraform within kubenow expects. (https://github.com/terraform-providers/terraform-provider-openstack/issues/275) If I'm not mistaken, this could be solved by updating the terraform dependency. However I will try to edit the openrc file as a workaround.

[andersla]

Thanks for investigating the issue! I will try merge a deployer docker image with updated Terraform version hopefully within this or beginning of next week.

[mcapuccini]

Good that you managed to sort this out! If you feel like you could share with us your modifications to the kn script, maybe with a PR.