Install Scripts for modern distributions

[h6w]

The install sequence is somewhat scattered across the Wiki, and the current README doesn't even list prerequisites for installation.

I think we should solve both these issues by writing scripts in POSIX shell (or WScript for Windows) that are fully commented and can be linked to to explain how an installation is expected to proceed on each platform that we "support".

e.g. RedHat/Fedora/CentOS Debian/Ubuntu/Mint Windows 7/10

We should also annotate each script with the systems they've been tested on so that we can keep track of what works and what doesn't.

After that, we should perhaps offer the user the option of pinging somewhere the lsb_release/windows release info and send feedback if/when it failed and what the error was.

[nils-ballmann]

I do have similar scripts built at work for Ubuntu. I can probably provide them as starting point on Tuesday.

[h6w]

I was about to do this mysellf, but that would be very helpful. Thanks!

[nils-ballmann]

I'm doing this on Ubuntu 16.04 LTS xenial server edition.

In setup_02_env.bash I install some functions in the bash environment:

function proxy_on() {
    export FTP_PROXY="<company proxy>"
    export HTTPS_PROXY="<company proxy>"
    export HTTP_PROXY="<company proxy>"
    export ftp_proxy="<company proxy>"
    export http_proxy="<company proxy>"
    export https_proxy="<company proxy>"
    echo "proxy export activated"
}
export -f proxy_on

function proxy_off() {
    unset HTTP_PROXY
    unset FTP_PROXY
    unset HTTPS_PROXY
    unset http_proxy
    unset ftp_proxy
    unset https_proxy
    echo "proxy export deactivated"
}
export -f proxy_off

Then VirtualBox is installed like in setup_05_vbox.bash.

#!/usr/bin/env bash

set -u
set -e
set -o pipefail

## depends on actions made in setup_02_env.bash [proxy_on/off]

echo ">>>> Importing VBox repo key <<<<"
proxy_on
wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
proxy_off

echo ">>>> Adding VBox repo <<<<"
echo -e '## Virtualbox\ndeb http://download.virtualbox.org/virtualbox/debian $(lsb_release -cs) contrib' | sudo tee /etc/apt/sources.list.d/virtualbox.list >/dev/null

echo ">>>> Updating repo index <<<<"
sudo apt update
echo ">>>> Installing build-essential <<<<"
sudo apt install build-essential

echo ">>>> Stop vbox services <<<<"
sudo systemctl --no-pager stop vboxdrv.service vboxautostart-service.service vboxweb-service.service || true

amount_of_time=10s
echo ">>>> Waiting $amount_of_time until all services shut down <<<<"
sleep "$amount_of_time"

echo ">>>> Installing VirtualBox 5.2 on non-x11 system <<<<"
sudo apt install --reinstall virtualbox-5.2 --no-install-recommends

echo ">>>> Adding vbox user <<<<"
sudo adduser --disabled-password --gecos "VirtualBox" vbox || true
echo ">>>> Setting password for vbox user <<<<"
pw=$(pwgen -s 30 1)
echo -e "${pw}\n${pw}\n" | sudo passwd vbox >/dev/null
echo "$pw" | sudo tee /root/vbox.user.password >/dev/null
sudo chmod og-rwx /root/vbox.user.password
echo ">>>> Adding vbox user to vboxusers group <<<<"
sudo adduser vbox vboxusers
echo ">>>> Adding vbox user to cdrom group <<<<"
sudo adduser vbox cdrom

vm_folder='/home/vbox/VBox/VMs'
echo ">>>> Creating '$vm_folder' <<<<"
sudo -i -u vbox mkdir -p "$vm_folder"
drives_folder='/home/vbox/VBox/drives'
echo ">>>> Creating '$drives_folder' <<<<"
sudo -i -u vbox mkdir -p "$drives_folder"
import_folder='/home/vbox/VBox/import'
echo ">>>> Creating '$import_folder' <<<<"
sudo -i -u vbox mkdir -p "$import_folder"
echo ">>>> Setting '$vm_folder' as default machine folder for user vbox <<<<"
sudo -i -u vbox vboxmanage setproperty machinefolder "$vm_folder"

echo ">>>> Removing any previous VBox Extension Pack <<<<"
sudo vboxmanage extpack uninstall "Oracle VM VirtualBox Extension Pack"

echo ">>>> Installing latest VBox Extension Pack <<<<"
cd /tmp
proxy_on
version=$(wget -qO- https://download.virtualbox.org/virtualbox/LATEST.TXT)
wget "https://download.virtualbox.org/virtualbox/${version}/Oracle_VM_VirtualBox_Extension_Pack-${version}.vbox-extpack"
proxy_off
sudo vboxmanage extpack install "Oracle_VM_VirtualBox_Extension_Pack-${version}.vbox-extpack"
rm "Oracle_VM_VirtualBox_Extension_Pack-${version}.vbox-extpack"
cd -

autostart_conf='/etc/vbox/autostart.conf'
echo ">>>> Creating '${autostart_conf}' <<<<"
echo -e "default_policy = deny\nvbox = {\n    allow = true\n}" | sudo tee "$autostart_conf" >/dev/null

autostart_folder='/home/vbox/VBoxAutostart'
echo ">>>> Creating '$autostart_folder' <<<<"
sudo -i -u vbox mkdir -p "$autostart_folder"
sudo -i -u vbox chmod g+w "$autostart_folder"
# set sticky bit
sudo -i -u vbox chmod +t "$autostart_folder"
sudo -i -u vbox vboxmanage setproperty autostartdbpath "$autostart_folder"

default_vbox='/etc/default/virtualbox'
echo ">>>> Removing old autostart config from '${default_vbox}' <<<<"
sudo touch "$default_vbox"
sudo sed -i -e "/^\s*VBOXAUTOSTART_DB\s*=.*$/d" "$default_vbox"
sudo sed -i -e "/^\s*VBOXAUTOSTART_CONFIG\s*=.*$/d" "$default_vbox"

echo ">>>> Appending new autostart config to '${default_vbox}' <<<<"
echo -e "VBOXAUTOSTART_DB=${autostart_folder}\nVBOXAUTOSTART_CONFIG=/etc/vbox/autostart.conf" | sudo tee -a "$default_vbox" >/dev/null

echo ">>>> Restarting vbox services <<<<"
sudo systemctl --no-pager status vboxdrv.service
sudo systemctl --no-pager restart vboxdrv.service
sudo systemctl --no-pager status vboxdrv.service

echo ">>>> Restarting vbox services <<<<"
sudo systemctl --no-pager status vboxautostart-service.service
sudo systemctl --no-pager restart vboxautostart-service.service
sudo systemctl --no-pager status vboxautostart-service.service

echo ">>>> Creating '/home/vbox/bin' <<<<"
sudo -i -u vbox mkdir -p "/home/vbox/bin"

echo ">>>> Creating '/home/vbox/bin/vbox_save_rvms.bash' <<<<"
set +e
read -r -d '' vbox_save_rvms_bash_content <<'EOF'
#!/usr/bin/env bash

set -u
set -e
set -o pipefail

svm="$HOME/saved_vms"
echo -n "" >"$svm"

vboxmanage list runningvms | cut -d'"' -f3 | cut -d" " -f2 | while read uid; do
  echo "Saving state of $uid ..."
  vboxmanage controlvm "$uid" savestate
  echo "DONE"
  echo "$uid" >>"$svm"
done

chmod og-rwx "$svm"
EOF
read_exit_val=$?
set -e
if [ $read_exit_val != 1 ]; then
    exit $read_exit_val
fi

echo "$vbox_save_rvms_bash_content" | sudo -i -u vbox tee /home/vbox/bin/vbox_save_rvms.bash >/dev/null
echo ">>>> Making '/home/vbox/bin/vbox_save_rvms.bash' executable <<<<"
sudo -i -u vbox chmod +x /home/vbox/bin/vbox_save_rvms.bash

echo ">>>> Creating '/home/vbox/bin/vbox_start_svms.bash' <<<<"
set +e
read -r -d '' vbox_start_svms_bash_content <<'EOF'
#!/usr/bin/env bash

set -u
set -e
set -o pipefail

svm="$HOME/saved_vms"

if [ ! -f "$svm" ]; then
  exit 0
fi

vms=$(vboxmanage list vms)
svms=$(cat "$svm")

for uid in $svms; do
  echo "Trying to start VM $uid"
  if [ $( echo "$vms" | grep "$uid" | wc -l ) == 0 ]; then
    echo "Found no VM $uid, skipping"
    continue
  else
    echo "Found VM $uid"
  fi

  echo "Starting VM $uid in headless mode..."
  vboxmanage startvm "$uid" --type headless
  echo "DONE"

  echo "Removing VM $uid from autostart"
  tmp_file=$(mktemp)
  grep -v "$uid" "$svm" >"$tmp_file" || true
  mv -f "$tmp_file" "$svm"
  echo "DONE"
done
EOF
read_exit_val=$?
set -e
if [ $read_exit_val != 1 ]; then
    exit $read_exit_val
fi

echo "$vbox_start_svms_bash_content" | sudo -i -u vbox tee /home/vbox/bin/vbox_start_svms.bash >/dev/null
echo ">>>> Making '/home/vbox/bin/vbox_start_svms.bash' executable <<<<"
sudo -i -u vbox chmod +x /home/vbox/bin/vbox_start_svms.bash

echo ">>>> Creating '/home/vbox/bin/vbox_activate_auto.bash' <<<<"
set +e
read -r -d '' vbox_activate_auto_bash_content <<'EOF'
#!/usr/bin/env bash

set -u
set -e
set -o pipefail

rvms=$(vboxmanage list runningvms)
vms=$(vboxmanage list vms)

function activate_auto() {
  if [ $( echo "$rvms" | grep "$1" | wc -l ) != 0 ]; then
    echo "VM $1 is running, skipping"
    continue
  fi

  if [ $( echo "$vms" | grep "$1" | wc -l ) == 0 ]; then
    echo "Found no VM $1, skipping"
    continue
  fi

  echo "Enabling autostart for VM $1 ..."
  vboxmanage modifyvm "$1" --autostart-enabled on
  echo "Enabling autostop (savestate) for VM $1 ..."
  vboxmanage modifyvm "$1" --autostop-type savestate
}

if [[ $# -eq 0 ]]; then
  echo "$vms" | cut -d'"' -f3 | cut -d" " -f2 | while read uid; do
    activate_auto "$uid"
  done
else
  while [ $# -gt 0 ]; do
    activate_auto "$1"
    shift
  done
fi
EOF
read_exit_val=$?
set -e
if [ $read_exit_val != 1 ]; then
    exit $read_exit_val
fi

echo "$vbox_activate_auto_bash_content" | sudo -i -u vbox tee /home/vbox/bin/vbox_activate_auto.bash >/dev/null
echo ">>>> Making '/home/vbox/bin/vbox_activate_auto.bash' executable <<<<"
sudo -i -u vbox chmod +x /home/vbox/bin/vbox_activate_auto.bash

echo ">>>> Creating '/home/vbox/bin/vbox_deactivate_auto.bash' <<<<"
set +e
read -r -d '' vbox_deactivate_auto_bash_content <<'EOF'
#!/usr/bin/env bash

set -u
set -e
set -o pipefail

rvms=$(vboxmanage list runningvms)
vms=$(vboxmanage list vms)

function deactivate_auto() {
  if [ $( echo "$rvms" | grep "$1" | wc -l ) != 0 ]; then
    echo "VM $1 is running, skipping"
    continue
  fi

  if [ $( echo "$vms" | grep "$1" | wc -l ) == 0 ]; then
    echo "Found no VM $1, skipping"
    continue
  fi

  echo "Disabeling autostart for VM $1 ..."
  vboxmanage modifyvm "$1" --autostart-enabled off
}

if [[ $# -eq 0 ]]; then
  echo "$vms" | cut -d'"' -f3 | cut -d" " -f2 | while read uid; do
    deactivate_auto "$uid"
  done
else
  while [ $# -gt 0 ]; do
    deactivate_auto "$1"
    shift
  done
fi
EOF
read_exit_val=$?
set -e
if [ $read_exit_val != 1 ]; then
    exit $read_exit_val
fi

echo "$vbox_deactivate_auto_bash_content" | sudo -i -u vbox tee /home/vbox/bin/vbox_deactivate_auto.bash >/dev/null
echo ">>>> Making '/home/vbox/bin/vbox_deactivate_auto.bash' executable <<<<"
sudo -i -u vbox chmod +x /home/vbox/bin/vbox_deactivate_auto.bash

echo ">>>> Changing aliases in '/home/vbox/.bashrc' <<<<"
sudo -i -u vbox sed -i -e "s/^alias ll=.*$/alias ll='ls -lh'/g" /home/vbox/.bashrc
sudo -i -u vbox sed -i -e "s/^alias la=.*$/alias la='ls -lah'/g" /home/vbox/.bashrc

The drives_folder is the folder, where the host-guest-shared-folders for the VMs will be, with the same hierarchy the VM has under /home/vbox/VBox/VMs/ (including group name and vm name).

The import_folder will be the folder, where the .ovas will be imported from.

I will restrict to those folders in the config.php.

[nils-ballmann]

Then in setup_06_phpVirtualBox.bash I install phpVirtualBox like this:

#!/usr/bin/env bash

set -u
set -e
set -o pipefail

## depends on actions made in setup_02_env.bash [proxy_on/off]
## depends on actions made in setup_04_vbox.bash [vbox user]

echo ">>>> Setting websrvauthlibrary <<<<"
sudo VBoxManage setproperty websrvauthlibrary null

default_vbox='/etc/default/virtualbox'
echo ">>>> Removing old vboxweb config from '${default_vbox}' <<<<"
sudo touch "$default_vbox"
sudo sed -i -e "/^\s*VBOXWEB_USER\s*=.*$/d" "$default_vbox"

echo ">>>> Appending new vboxweb config to '${default_vbox}' <<<<"
echo "VBOXWEB_USER=vbox" | sudo tee -a /etc/default/virtualbox >/dev/null

phpvbox='/opt/phpvirtualbox'

hname=$(hostname)
# remove leading whitespace characters
hname="${hname#"${hname%%[![:space:]]*}"}"
# remove trailing whitespace characters
hname="${hname%"${hname##*[![:space:]]}"}"

fqdn=$(hostname -A)
# remove leading whitespace characters
fqdn="${fqdn#"${fqdn%%[![:space:]]*}"}"
# remove trailing whitespace characters
fqdn="${fqdn%"${fqdn##*[![:space:]]}"}"

echo ">>>> Removing previous $phpvbox <<<<"
sudo rm -rf "$phpvbox"
echo ">>>> Creating $phpvbox <<<<"
sudo mkdir "$phpvbox"
echo ">>>> Owning by administrator <<<<"
sudo chown administrator:vboxusers "$phpvbox"

echo ">>>> Cloning phpvirtualbox <<<<"
proxy_on
git clone https://github.com/phpvirtualbox/phpvirtualbox.git "$phpvbox"
proxy_off

sudo chown -R administrator:vboxusers "$phpvbox"

cd "$phpvbox"
echo ">>>> Using develop branch <<<<"
git checkout develop

echo ">>>> Changing path to $phpvbox in phpvirtualbox.conf <<<<"
sed -i -e "s/\/usr\/share/\/opt/g" phpvirtualbox.conf
echo ">>>> Allowing connections from anywhere in phpvirtualbox.conf <<<<"
sed -i -e "s/^\(\s*\)Require local$/\1#Require local\n\1# Allow connections from anywhere:\nRequire all granted/g" phpvirtualbox.conf
echo ">>>> Changing alias in phpvirtualbox.conf <<<<"
sed -i -e "s/Alias \/phpvirtualbox\S*/Alias \/phpvirtualbox.${hname}/g" phpvirtualbox.conf

dpkg -s apache2 php7.0 libapache2-mod-php7.0 >/dev/null 2>&1
if [ $? -ne 0 ]; then
    echo "Dependency is missing:"
    apt-cache policy apache2 php7.0 libapache2-mod-php7.0 | grep -B1 Installed
    exit 1
fi

echo ">>>> Installing other dependencies <<<<"
sudo apt install php7.0-soap php7.0-xml

echo ">>>> Installing apache alias <<<<"
sudo cp phpvirtualbox.conf /etc/apache2/conf-available/
echo ">>>> Enabeling apache alias <<<<"
sudo a2enconf phpvirtualbox
echo ">>>> Reloading apache config <<<<"
sudo systemctl --no-pager reload apache2

echo ">>>> Creating phpVirtualbox config <<<<"
if sudo [ ! -f /root/vbox.user.password ]; then
    echo "Dependency is missing: /root/vbox.user.password"
    exit 1
fi
cp config.php-example config.php

echo ">>>> Setting user and password in phpVirtualbox config <<<<"
sed -i -e 's/^var $username = .*$/var $username = '"'vbox'"';/g' config.php
pw=$(sudo cat /root/vbox.user.password)
sed -i -e 's/^var $password = .*$/var $password = '"'${pw}'"';/g' config.php

echo ">>>> Setting site name in phpVirtualbox config <<<<"
sed -i -e 's/^\(var $location = .*\)$/\1\nvar $name = '"'${fqdn}'"';/g' config.php

echo ">>>> Setting folder restrictions in phpVirtualbox config <<<<"
sed -i -e 's/^\s*#*\s*\(var $browserRestrictFolders\)\s*=\s*.*$/\1 = array('"'\/home\/vbox\/VBox'"');/g' config.php

echo ">>>> Disable VMConsole in phpVirtualbox config <<<<"
sed -i -e 's/^\s*#*\s*\(var $disableTabVMConsole =.*\)$/\1/g' config.php

echo ">>>> Setting Console Keyboard Layout in phpVirtualbox config <<<<"
sed -i -e 's/^\s*#*\s*\(var $consoleKeyboardLayout = '"'\)EN\('"'.*\)$/\1DE\2/g' config.php

echo ">>>> Enabling advanced vm config in phpVirtualbox config <<<<"
sed -i -e 's/^\s*#*\s*\(var $enableAdvancedConfig =.*\)$/\1/g' config.php

echo ">>>> Restarting vboxweb services <<<<"
sudo systemctl --no-pager status vboxweb-service.service
sudo systemctl --no-pager restart vboxweb-service.service
sudo systemctl --no-pager status vboxweb-service.service
sudo netstat -tulpena | grep 18083 || true

I disable the VMConsole because we use Guacamole to let the users connect to their VMs and because I don't want to use flash.

[nils-ballmann]

All those scripts are for attended installation.

[h6w]

@nils-ballmann That's great. Thanks! I started modifying these and I'm at odds to understand why you need proxy_(on|off) functions rather than just assigning the variables before the script is run. Does your proxy not handle normal apt requests?

[nils-ballmann]

I put things into functions to escape copy-paste-hell. And I activate the proxy just before I use it (e.g. with wget) and deactivate it directly after. This way I just have an internet-connection where I need it.

In non proxy setups they are unnecessary. But I kept them in, to have a reminder, that proxies may be an issue.

apt itself has its proxy configuration in /etc/apt/apt.conf.

[h6w]

@nils-ballmann Yes, but why on-script-off-script-on-script-off-script when you can just do on-script-script-script-off?

[nils-ballmann]

Yes you can do it like that. Would be easier to write. But I as the author want to be aware of the script making a call to a remote location.

[stefanhorning]

For people prefering provisioning tools instead of bash scripts, I wrote this Ansible role a while back, which I recently updated to work better: https://github.com/mediapeers/ansible-role-virtualbox

Might also be starting point to develop other scripts.

[nils-ballmann]

Good call!

As we are using saltstack, maybe there will be a salt-receipt too, some day.