osie symlinks are breaking this sandbox

JamesPGriffith commented 3 years ago

Expected Behaviour

The sandbox provisioner starts.

Current Behaviour

The sandbox provisioner fails to start.

Possible Solution

Unknown at the moment.

Steps to Reproduce (for bugs)

Follow the steps here: https://docs.tinkerbell.org/setup/local-vagrant/

Context

I am unable to test tinkerbell in the pre-designed and isolated vagrantbox.

Your Environment

Operating System and version (e.g. Linux, Windows, MacOS):
Windows 10
How are you running Tinkerbell? Using Vagrant & VirtualBox, Vagrant & Libvirt, on Packet using Terraform, or give details:
Vagrant

Link to your project or a code example to reproduce issue:
I added a a few commands to indicate the PWD and the contents of PWD to determine that symlinks are an issue.

provisioner: + curl https://tinkerbell-oss.s3.amazonaws.com/osie-uploads/osie-v0-n=404,c=c35a5f8,b=master.tar.gz -o ./osie.tar.gz
provisioner:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
provisioner:                                  Dload  Upload   Total   Spent    Left  Speed
50 2174M   50 1107M    0     0  17.7M      0  0:02:02  0:01:01  0:01:01 15.9M
96 2174M   96 2100M    0     0  16.7M      0  0:02:09  0:02:05  0:00:04 17.1M
100 2174M  100 2174M    0     0  16.8M      0  0:02:08  0:02:08 --:--:-- 21.0M
provisioner: + tar -zxf osie.tar.gz
provisioner: + pushd osie-v0-n=404,c=c35a5f8,b=master/
provisioner: /tmp/tmp.20jmEiTnhL/osie-v0-n=404,c=c35a5f8,b=master /tmp/tmp.20jmEiTnhL /vagrant
provisioner: + mv workflow-helper.sh workflow-helper-rc /vagrant/deploy/state/webroot/workflow/
provisioner: ++ pwd
provisioner: THE PWD IS /tmp/tmp.20jmEiTnhL/osie-v0-n=404,c=c35a5f8,b=master
provisioner: + echo 'THE PWD IS /tmp/tmp.20jmEiTnhL/osie-v0-n=404,c=c35a5f8,b=master'
provisioner: + ls -la ./discover-metal-x86_64.tar.gz ./discover-rc ./discover.sh ./grub ./initramfs-2a2 ./initramfs-aarch64 ./initramfs-amp ./initramfs-hua ./initramfs-qcom ./initramfs-tx2 ./initramfs-x86_64 ./modloop-2a2 ./modloop-aarch64 ./modloop-amp ./modloop-hua ./modloop-qcom ./modloop-tx2 ./modloop-x86_64 
./osie-aarch64.tar.gz ./osie-installer-rc ./osie-installer.sh ./osie-runner-x86_64.tar.gz ./osie-x86_64.tar.gz ./repo-aarch64 ./repo-x86_64 ./rescue-helper-rc ./rescue-helper.sh ./runner-rc ./runner.sh ./vmlinuz-2a2 ./vmlinuz-aarch64 ./vmlinuz-amp ./vmlinuz-hua ./vmlinuz-qcom ./vmlinuz-tx2 ./vmlinuz-x86_64
provisioner: -rw-r--r--  1 root root 1063704576 Jan  6  2021 ./discover-metal-x86_64.tar.gz
provisioner: -rw-r--r--  1 root root        133 Jan  6  2021 ./discover-rc
provisioner: -rw-r--r--  1 root root       2499 Jan  6  2021 ./discover.sh
provisioner: -rw-r--r--  1 root root   41651322 Jan  6  2021 ./initramfs-2a2
provisioner: -rw-r--r--  1 root root   45498385 Jan  6  2021 ./initramfs-aarch64
provisioner: -rw-r--r--  1 root root   42696062 Jan  6  2021 ./initramfs-amp
provisioner: -rw-r--r--  1 root root   37430247 Jan  6  2021 ./initramfs-hua
provisioner: -rw-r--r--  1 root root   42708137 Jan  6  2021 ./initramfs-qcom
provisioner: -rw-r--r--  1 root root   49447343 Jan  6  2021 ./initramfs-tx2
provisioner: -rw-r--r--  1 root root  130723463 Jan  6  2021 ./initramfs-x86_64
provisioner: -rw-r--r--  1 root root   39305216 Jan  6  2021 ./modloop-2a2
provisioner: -rw-r--r--  1 root root  188854272 Jan  6  2021 ./modloop-aarch64
provisioner: -rw-r--r--  1 root root   40345600 Jan  6  2021 ./modloop-amp
provisioner: -rw-r--r--  1 root root   35037184 Jan  6  2021 ./modloop-hua
provisioner: -rw-r--r--  1 root root   40349696 Jan  6  2021 ./modloop-qcom
provisioner: -rw-r--r--  1 root root   47124480 Jan  6  2021 ./modloop-tx2
provisioner: -rw-r--r--  1 root root  209412096 Jan  6  2021 ./modloop-x86_64
provisioner: -rw-r--r--  1 root root  706898432 Jan  6  2021 ./osie-aarch64.tar.gz
provisioner: -rw-r--r--  1 root root         96 Jan  6  2021 ./osie-installer-rc
provisioner: -rw-r--r--  1 root root       7003 Jan  6  2021 ./osie-installer.sh
provisioner: -rw-r--r--  1 root root  423728640 Jan  6  2021 ./osie-runner-x86_64.tar.gz
provisioner: -rw-r--r--  1 root root  892384256 Jan  6  2021 ./osie-x86_64.tar.gz
provisioner: lrwxrwxrwx  1 root root         20 Jan  6  2021 ./repo-aarch64 -> ../../../alpine/edge
provisioner: lrwxrwxrwx  1 root root         21 Jan  6  2021 ./repo-x86_64 -> ../../../alpine/v3.12
provisioner: -rw-r--r--  1 root root        123 Jan  6  2021 ./rescue-helper-rc
provisioner: -rw-r--r--  1 root root       1019 Jan  6  2021 ./rescue-helper.sh
provisioner: -rw-r--r--  1 root root         69 Jan  6  2021 ./runner-rc
provisioner: -rw-r--r--  1 root root       4764 Jan  6  2021 ./runner.sh
provisioner: -rw-r--r--  1 root root   10412544 Jan  6  2021 ./vmlinuz-2a2
provisioner: -rw-r--r--  1 root root   14852608 Jan  6  2021 ./vmlinuz-aarch64
provisioner: -rw-r--r--  1 root root   11272704 Jan  6  2021 ./vmlinuz-amp
provisioner: -rw-r--r--  1 root root    8912384 Jan  6  2021 ./vmlinuz-hua
provisioner: -rw-r--r--  1 root root   11270656 Jan  6  2021 ./vmlinuz-qcom
provisioner: -rw-r--r--  1 root root   11270656 Jan  6  2021 ./vmlinuz-tx2
provisioner: -rw-r--r--  1 root root    6699168 Jan  6  2021 ./vmlinuz-x86_64
provisioner:
provisioner: ./grub:
provisioner: total 92
provisioner: drwxr-xr-x 23 root root 4096 Jul 25 03:02 .
provisioner: drwxr-xr-x  3 root root 4096 Jul 25 03:03 ..
provisioner: drwxr-xr-x 35 root root 4096 Jul 25 03:02 centos_7
provisioner: drwxr-xr-x 36 root root 4096 Jul 25 03:02 centos_8
provisioner: drwxr-xr-x 31 root root 4096 Jul 25 03:02 debian_10
provisioner: drwxr-xr-x 30 root root 4096 Jul 25 03:02 debian_8
provisioner: drwxr-xr-x 34 root root 4096 Jul 25 03:02 debian_9
provisioner: drwxr-xr-x 33 root root 4096 Jul 25 03:02 opensuse_42_3
provisioner: drwxr-xr-x 31 root root 4096 Jul 25 03:02 rhel_7
provisioner: drwxr-xr-x 31 root root 4096 Jul 25 03:02 rhel_8
provisioner: drwxr-xr-x 27 root root 4096 Jul 25 03:02 scientific_6
provisioner: drwxr-xr-x 24 root root 4096 Jul 25 03:02 suse_sles12_sp3
provisioner: drwxr-xr-x 29 root root 4096 Jul 25 03:02 ubuntu_14_04
provisioner: drwxr-xr-x 40 root root 4096 Jul 25 03:02 ubuntu_16_04
provisioner: drwxr-xr-x 32 root root 4096 Jul 25 03:02 ubuntu_17_04
provisioner: drwxr-xr-x 32 root root 4096 Jul 25 03:02 ubuntu_17_10
provisioner: drwxr-xr-x 40 root root 4096 Jul 25 03:02 ubuntu_18_04
provisioner: drwxr-xr-x 39 root root 4096 Jul 25 03:02 ubuntu_19_04
provisioner: drwxr-xr-x 39 root root 4096 Jul 25 03:02 ubuntu_19_10
provisioner: drwxr-xr-x 40 root root 4096 Jul 25 03:02 ubuntu_20_04
provisioner: drwxr-xr-x 40 root root 4096 Jul 25 03:02 ubuntu_20_10
provisioner: drwxr-xr-x 40 root root 4096 Jul 25 03:02 vmware_nsx_2_5_0
provisioner: drwxr-xr-x 40 root root 4096 Jul 25 03:02 vmware_nsx_3_0_0
provisioner: + cp -r ./discover-metal-x86_64.tar.gz ./discover-rc ./discover.sh ./grub ./initramfs-2a2 ./initramfs-aarch64 ./initramfs-amp ./initramfs-hua ./initramfs-qcom ./initramfs-tx2 ./initramfs-x86_64 ./modloop-2a2 ./modloop-aarch64 ./modloop-amp ./modloop-hua ./modloop-qcom ./modloop-tx2 ./modloop-x86_64 ./osie-aarch64.tar.gz ./osie-installer-rc ./osie-installer.sh ./osie-runner-x86_64.tar.gz ./osie-x86_64.tar.gz ./repo-aarch64 ./repo-x86_64 ./rescue-helper-rc ./rescue-helper.sh ./runner-rc ./runner.sh ./vmlinuz-2a2 ./vmlinuz-aarch64 ./vmlinuz-amp ./vmlinuz-hua ./vmlinuz-qcom ./vmlinuz-tx2 ./vmlinuz-x86_64 /vagrant/deploy/state/webroot/misc/osie/current
provisioner: cp: cannot create symbolic link '/vagrant/deploy/state/webroot/misc/osie/current/repo-aarch64': Protocol error
provisioner: cp: cannot create symbolic link '/vagrant/deploy/state/webroot/misc/osie/current/repo-x86_64': Protocol error
provisioner: + finish
provisioner: + rm -rf /tmp/tmp.20jmEiTnhL
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.
PS C:\Users\user\projects\sandbox\deploy\vagrant>

JamesPGriffith commented 3 years ago

Running this on macOS is a little more graceful. Odds are, any *NIX OS will handle this better. I am using this branch to get past #91.

user@MacBook-Pro vagrant % vagrant destroy --force ; vagrant up provisioner
[[TRANCATED]]
    provisioner: + tar -zxf osie.tar.gz
    provisioner: /tmp/tmp.yecpwZ9p0N/osie-v0-n=404,c=c35a5f8,b=master /tmp/tmp.yecpwZ9p0N /vagrant
    provisioner: + pushd osie-v0-n=404,c=c35a5f8,b=master/
    provisioner: + mv workflow-helper.sh workflow-helper-rc /vagrant/deploy/state/webroot/workflow/
    provisioner: + cp -r ./discover-metal-x86_64.tar.gz ./discover-rc ./discover.sh ./grub ./initramfs-2a2 ./initramfs-aarch64 ./initramfs-amp ./initramfs-hua ./initramfs-qcom ./initramfs-tx2 ./initramfs-x86_64 ./modloop-2a2 ./modloop-aarch64 ./modloop-amp ./modloop-hua ./modloop-qcom ./modloop-tx2 ./modloop-x86_64 ./osie-aarch64.tar.gz ./osie-installer-rc ./osie-installer.sh ./osie-runner-x86_64.tar.gz ./osie-x86_64.tar.gz ./repo-aarch64 ./repo-x86_64 ./rescue-helper-rc ./rescue-helper.sh ./runner-rc ./runner.sh ./vmlinuz-2a2 ./vmlinuz-aarch64 ./vmlinuz-amp ./vmlinuz-hua ./vmlinuz-qcom ./vmlinuz-tx2 ./vmlinuz-x86_64 /vagrant/deploy/state/webroot/misc/osie/current
    provisioner: /tmp/tmp.yecpwZ9p0N /vagrant
    provisioner: + popd
    provisioner: + generate_certificates
[[TRANCATED]]
user@MacBook-Pro vagrant % vagrant ssh provisioner                         
vagrant@provisioner:~$ find . -xtype l
vagrant@provisioner:~$ cd /vagrant
vagrant@provisioner:/vagrant$ find . -xtype l
./deploy/state/webroot/misc/osie/current/repo-aarch64
./deploy/state/webroot/misc/osie/current/repo-x86_64
vagrant@provisioner:/vagrant$

A possible paths:

Create the osie tar following the symlinks.
Don't untar into the host OS filesystem.

I lean toward both. I'm still exploring the application so I don't have any code to contribute at the moment.

displague commented 3 years ago

@JamesPGriffith feel free to open a PR with https://github.com/tinkerbell/sandbox/compare/master...JamesPGriffith:91-TINKERBELL_SKIP_NETWORKING-unbound-variable?expand=1

Any tarball that depends on symlinks it creates, outside of tar root is dangerous.

Running this on macOS is a little more graceful

This could be a difference between the default behaviors of GNU and BSD tar.

I would expect that the tar would be executed within the vagrant environment. While slower, this would guarantee that the paths are correct and do not pose a chance of disturbing files on the host. (guarantee -- only if a host filesystem overlay or host-nfs is not used).

jacobweinstock commented 3 years ago

Hey @JamesPGriffith, #90 just landed and has changed the sandbox significantly. Would you mind having a look at the changes and confirm if you are still experiencing this behavior?

JamesPGriffith commented 3 years ago

Interestingly, I'm not even getting this far. It looks like a Vagrant issue or perhaps an issue with the box template?

james@MacBook-Pro ~ % neofetch
                    'c.          james@MacBook-Pro.home.griffith.gq 
                 ,xNMM.          ---------------------------------- 
               .OMMMMo           OS: macOS 11.5.2 20G95 x86_64 
               OMMM0,            Host: MacBookPro11,2 
     .;loddo:' loolloddol;.      Kernel: 20.6.0 
   cKMMMMMMMMMMNWMMMMMMMMMM0:    Uptime: 2 days, 16 hours, 36 mins 
 .KMMMMMMMMMMMMMMMMMMMMMMMWd.    Packages: 99 (brew) 
 XMMMMMMMMMMMMMMMMMMMMMMMX.      Shell: zsh 5.8 
;MMMMMMMMMMMMMMMMMMMMMMMM:       Resolution: 1440x900 
:MMMMMMMMMMMMMMMMMMMMMMMM:       DE: Aqua 
.MMMMMMMMMMMMMMMMMMMMMMMMX.      WM: Quartz Compositor 
 kMMMMMMMMMMMMMMMMMMMMMMMMWd.    WM Theme: Blue (Dark) 
 .XMMMMMMMMMMMMMMMMMMMMMMMMMMk   Terminal: vscode 
  .XMMMMMMMMMMMMMMMMMMMMMMMMK.   CPU: Intel i7-4850HQ (8) @ 2.30GHz 
    kMMMMMMMMMMMMMMMMMMMMMMd     GPU: Intel Iris Pro 
     ;KMMMMMMMWXXWMMMMMMMk.      Memory: 10252MiB / 16384MiB 
       .cooc,.    .,coo:.

james@MacBook-Pro ~ % cd ~/projects
james@MacBook-Pro projects % rm -rf sandbox
james@MacBook-Pro projects % git clone https://github.com/tinkerbell/sandbox.git
Cloning into 'sandbox'...
remote: Enumerating objects: 732, done.
remote: Counting objects: 100% (414/414), done.
remote: Compressing objects: 100% (214/214), done.
remote: Total 732 (delta 213), reused 316 (delta 164), pack-reused 318
Receiving objects: 100% (732/732), 856.03 KiB | 5.42 MiB/s, done.
Resolving deltas: 100% (317/317), done.
james@MacBook-Pro sandbox % git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
james@MacBook-Pro sandbox % cd deploy/vagrant/
james@MacBook-Pro vagrant % vagrant version
Installed Version: 2.2.18
Latest Version: 2.2.18

You're running an up-to-date version of Vagrant!
james@MacBook-Pro vagrant % vagrant up
Bringing machine 'provisioner' up with 'virtualbox' provider...
==> provisioner: Importing base box 'generic/ubuntu2004'...
==> provisioner: Matching MAC address for NAT networking...
==> provisioner: Checking if box 'generic/ubuntu2004' version '3.4.2' is up to date...
==> provisioner: Setting the name of the VM: vagrant_provisioner_1630767993677_48783
==> provisioner: Fixed port collision for 22 => 2222. Now on port 2201.
==> provisioner: Clearing any previously set network interfaces...
==> provisioner: Preparing network interfaces based on configuration...
    provisioner: Adapter 1: nat
    provisioner: Adapter 2: hostonly
==> provisioner: Forwarding ports...
    provisioner: 22 (guest) => 2201 (host) (adapter 1)
==> provisioner: Running 'pre-boot' VM customizations...
==> provisioner: Booting VM...
==> provisioner: Waiting for machine to boot. This may take a few minutes...
    provisioner: SSH address: 127.0.0.1:2201
    provisioner: SSH username: vagrant
    provisioner: SSH auth method: private key
    provisioner: 
    provisioner: Vagrant insecure key detected. Vagrant will automatically replace
    provisioner: this with a newly generated keypair for better security.
    provisioner: 
    provisioner: Inserting generated public key within guest...
    provisioner: Removing insecure key from the guest if it's present...
    provisioner: Key inserted! Disconnecting and reconnecting using new SSH key...
==> provisioner: Machine booted and ready!
==> provisioner: Checking for guest additions in VM...
==> provisioner: Mounting shared folders...
    provisioner: /vagrant => /Users/james/projects/sandbox/deploy
Vagrant was unable to mount VirtualBox shared folders. This is usually
because the filesystem "vboxsf" is not available. This filesystem is
made available via the VirtualBox Guest Additions and kernel module.
Please verify that these guest additions are properly installed in the
guest. This is not a bug in Vagrant and is usually caused by a faulty
Vagrant box. For context, the command attempted was:

mount -t vboxsf -o uid=1000,gid=1000,_netdev vagrant /vagrant

The error output from the command was:

: Invalid argument

james@MacBook-Pro vagrant %

I'm going to close this issue though as it's likely resolved or no longer relevant.

tinkerbell / playground