Bare-metal Linux VServer preparation script for BOA container

[macmladen]

Can you share some preparatory script that prepares bare-metal and puts up VServer to host BOA?

Also I'd be grateful if you can provide some practical tutorial (link) that could help me in the process as I was wondering how can I have BOA and Redmine, Gitlab, Mail all of which may run in their own containers (VServers).

I was looking for some solution to have Redmine and/or mail in separate containers so I wanted LXC as it seemed lightweight but then I was wondering if Docker is just toolset for LXC or does it offer more than just that.

972

It's a good idea to open source this tiny and simple tool we use to set up VServer containers.

[macmladen]

As we started discussion in #972 I was wondering on virtualization techniques.

Debian was supporting linux-vserver but now it is deprecated,

Linux-VServer support is no longer present in Debian, starting from wheezy.

There are, let's say, three main virtualization techniques (or approaches based on needs):

• Virtual machines (VMs) (kvm, qemu, vmware, virtualbox, virtuozzo, hyper-v)
• Paravirtualization (xen, uml)
• Virtualization on the OS level, a.k.a. containers virtualization

Linux VServer belongs to the third group which is probably the best for most use cases as that group presents the least overhead (in range from 1% to 3%). There are several operating system level virtualization implementations:

• Virtuozzo / OpenVZ (2000/2005)
• Linux VServer (2001)
• LXC linux containers (2008)
• Docker (2013)

OpenVZ required custom kernel and today is directly supported on CentOS. Linux VServer was supported on Debian before Wheezy. Docker is supposed to be used only for one process so it is not suited well for complete BOA stack.

Based on similarity between Linux VServer and LXC (which seem like more modern, current approach to virtualization), is there some specific reason not to go with LXC?

[omega8cc]

LXC is more complex to use, and got into kernel not because it is better or more modern than Linux VServer. It happened because of politics, better personal connections, etc. All that stuff which is always a warning sign for us. That is why we prefer Linux VServer, which is actively maintained despite being replaced upstream with LXC.

[omega8cc]

Also, LXC is supported by Canonical Ltd. and we don't like that fact, either. Plus, it is still largely a work in progress: https://linuxcontainers.org

[EdNett]

Hello, I am to understand then that BOA officially recommends linux-verser, which has been deprecated by debian in jessie? IS it being actively maintained on github - is that the place you suggest we take it from?

[omega8cc]

@EdNett -- We don't care what Debian mainstream kernel offers. If you are interested in VServer kernel details, please check:

http://linux-vserver.org/Frequently_Asked_Questions http://linux-vserver.org/Communicate http://www.psand.net/repo/

That said, don't ask us for help in installing it on your own. You should definitely not try this if you are not fluent in custom kernel management, broken grub config repair, broken machines recovery etc. Otherwise you can screw up things royally, and it is better to wait until installing VServer will be supported by BOA tools.

[omega8cc]

New installer has been added: https://github.com/omega8cc/boa/blob/master/docs/CLUSTER.txt#L77

It is a work in progress! For testing only, for now.

[ar-jan]

I'm unlikely to use this personally (anytime soon, anyway), but it's great to see this added. Thanks!

[angel0d]

@omega8cc tested the installed last night and everything seems to have worked fine, but ...

worked fine:

   $ cluster in-host server.example.com
   $ reboot
   $ cluster up-host
   $ cluster in-all head

it seems that there is no mySQL installed in the 3 vservers c1rdb0 c1rdb1 c1rdb2

c1rweb works fine with its local DB

but adding an additional octopus instance did not work $ cluster in-oct em@il o2 mini head

with error:

BOA [13:15:02] ==> INIT A: Shared platforms code v.001 will be used
BOA [13:15:02] ==> INIT A: Adding user...
mysqladmin: flush failed; error: 'Max connect timeout reached while reaching hostgroup 11 after 10000ms'
mysqladmin: reload failed; error: 'Max connect timeout reached while reaching hostgroup 11 after 10000ms'
SQL14 -h127.0.0.1 -P6033 --protocol=tcp -uroot
ERROR 9001 (HY000) at line 1: Max connect timeout reached while reaching hostgroup 11 after 10000ms
mysqladmin: reload failed; error: 'Max connect timeout reached while reaching hostgroup 11 after 10000ms'
BOA [13:15:59] ==> INIT B: Downloading drush micro-8-07-02-2017...
BOA [13:16:02] ==> INIT B: Installing Aegir Provision backend...
BOA [13:16:02] ==> INIT B: Downloading Drush and Provision extensions...
BOA [13:16:11] ==> INIT B: Running hostmaster-install, please wait...
/opt/tmp/boa/lib/functions/satellite.sh.inc: line 4765: cd: /data/disk/sol/aegir/distro/001: No such file or directory
BOA [13:16:25] ==> INIT B: FATAL ERROR: Required file does not exist:
BOA [13:16:25] ==> INIT B: FATAL ERROR: /data/disk/sol/aegir/distro/001/sites/sol.cluster.solebich.de/settings.php
BOA [13:16:25] ==> INIT B: FATAL ERROR: Aborting AegirSetupB installer NOW!
BOA [13:16:25] ==> INIT A: FATAL ERROR: AegirSetupB installer failed
BOA [13:16:25] ==> INIT A: FATAL ERROR: Aborting AegirSetupA installer NOW!
BOA [13:16:25] ==> FATAL ERROR: AegirSetupA installer failed
BOA [13:16:25] ==> FATAL ERROR: Aborting Octopus installer NOW!

boa info on c1rweb : https://gist.github.com/angel0d/ede19c4b4ac59943f6516aca595e4362

host is a dedicated on leaseweb with the following specs

Brand and Model Hp - DL120 G6
Processor   1x Intel Quad-Core Xeon X3440 [ 4 Core(s) ]
RAM 16 GB
HDD 2x120 GB SSD 2.5 Inch
Hardware RAID   No

I understand this issue regards the in-vps part of the installation script. maybe #1136 or #502 would be more appropriate for feedback on the cluster part of the script ?

[omega8cc]

Yeah, there are two patches we forgot to apply, sorry about that! Should be fixed today.

[angel0d]

@omega8cc is it correct to assume that deleting the vservers would allow a new cluster in-all head to take place ?

or re-installing the host OS is a must ?

[omega8cc]

@angel0d You don't need to re-install anything, not even the guests. Once the patches are committed, it will be enough to re-install just the ProxySQL on the web node with:

cluster {in-pxy} {id} {ip}

which translates to:

cluster in-pxy c1rweb your-web-guest-ip

[omega8cc]

As for re-installing the guests (which is not needed to fix this particular problem), yes, you don't need to re-install the host, just run this script on the host:

#!/bin/bash

export DEBIAN_FRONTEND=noninteractive

/usr/sbin/vserver --silent c1rdb0 stop
sleep 3
/usr/sbin/vserver --silent c1rdb0 delete
sleep 3
chattr -i -R /vservers/c1rdb0
rm -f -r /vservers/c1rdb0

/usr/sbin/vserver --silent c1rdb1 stop
sleep 3
/usr/sbin/vserver --silent c1rdb1 delete
sleep 3
chattr -i -R /vservers/c1rdb1
rm -f -r /vservers/c1rdb1

/usr/sbin/vserver --silent c1rdb2 stop
sleep 3
/usr/sbin/vserver --silent c1rdb2 delete
sleep 3
chattr -i -R /vservers/c1rdb2
rm -f -r /vservers/c1rdb2

/usr/sbin/vserver --silent c1rweb stop
sleep 3
/usr/sbin/vserver --silent c1rweb delete
sleep 3
chattr -i -R /vservers/c1rweb
rm -f -r /vservers/c1rweb

find /vservers/.hash -type f -links 1 -exec rm -v '{}' ';'

[omega8cc]

@angel0d The fix has been committed, but it should be enough to run this simple script to get it fixed: https://gist.github.com/omega8cc/3b1fb183b7b6fcb1c8c5f56e36d0e8a7

[angel0d]

@omega8cc just run the script but not sure how it could help

it seems to me that there is no mysql installed in the DB vservers

root@cluster:~# vserver c1rdb1 enter
root@c1rdb1:/# mysql
bash: mysql: command not found
root@c1rdb1:/# ps -A | grep mysql
root@c1rdb1:/# 

I 'll re-try to re-install the vservers - is there a specfific log file that could help debug this further ?

[omega8cc]

@angel0d We have left most of the output to be printed on screen, it doesn't log anything in this initial version.

You should see errors if it couldn't install something.

We have tested it many times, and cluster in-all head just worked. No issues...

[omega8cc]

@angel0d Please check twice that the IPs are correctly listed and that they can be enabled by VServer (they should be assigned to the machine already, but don't need to be active before the installation).

[omega8cc]

@angel0d Please use newer version of the script, we have reduced default noise cancellation level from -qq to -q

[angel0d]

We have left most of the output to be printed on screen, it doesn't log anything in this initial version.

I noticed that. will do the next install in screen so as to have all of it there for inspection

Please check twice that the IPs are correctly listed and that they can be enabled by VServer

I assumed that they should be enabled and working beforehand and I have added all public and private IPs as aliases on the primary interface

[angel0d]

btw I noticed that cluster up-all head seems to do nothing and returns in a second

[omega8cc]

@angel0d Please don't enable the IPs, they should be assigned, but not enabled.

[omega8cc]

@angel0d

btw I noticed that cluster up-all head seems to do nothing and returns in a second

That's correct:

upgrade_db_cluster() {
  check_config_cluster
  ### TODO
}

upgrade_web_node() {
  check_config_cluster
  ### TODO
}

[angel0d]

so I removed the IP aliases

after the base install on 1st vserver when it moves on with the galera setup it is not that verbose

I only got the following

Running post-install Galera VPS upgrade...
Removing systemd on jessie...
The c1rdb0 VPS installation is complete!
Installing Percona Cluster on VPS c1rdb0...
Warning: Permanently added '192.30.1.193' (ECDSA) to the list of known hosts.
bash: line 12: /etc/mysql/debian.cnf: No such file or directory
Warning: Permanently added '192.30.1.193' (ECDSA) to the list of known hosts.
bash: line 49: /etc/mysql/conf.d/galera.cnf: No such file or directory
Warning: Permanently added '192.30.1.193' (ECDSA) to the list of known hosts.
bash: line 113: /etc/mysql/my.cnf: No such file or directory
Warning: Permanently added '192.30.1.193' (ECDSA) to the list of known hosts.
mysql: unrecognized service

[angel0d]

although it installed the base system fine it seems that the c1rdb0 does not have outgoing access to the net

[omega8cc]

That could be your system/network issue, I guess?

[angel0d]

That could be your system/network issue, I guess?

i am trying to debug this but with no luck. What are the assumptions made for the private network ?

Is it supposed to have routed public access already ? Or should the host be set up so as to act as a NAT gateway ?

in my case private network is configured for a specific subnet. I see in cluster.sh

  vserver ${idn} build -n ${idn} -m debootstrap --i-know-its-there \
    --hostname ${hst} \
    --interface eth0:${vip}/32 \
    -- -d ${osx} -- --arch=amd64 --include=${incl} \
    --exclude=${excl}; &> /dev/null

maybe the prefix should be configurable as different providers/datacenters might have specific network settings ?

i tried to change the above line so as to be in accordance to the private network configuration (which is 10.30.1.192/27), use the clean script and re-run in-all but config in /usr/etc/vservers/c1rdb0/interfaces/0/ seems to be the same

[omega8cc]

The /32 netmask is not network specific, because it means "just this single IP address", and it works everywhere, no matter the server provider. At least, it works everywhere we have used it, at various data centers in the last 12 years. Example.

Thats said, the script was not tested with local IPs configured on interface other than eth0 and this perhaps should be configurable, at least for db nodes, because web node must use public IP address, no matter what.

Further testing and patches are welcome!

[omega8cc]

The /32 netmask should just work also with private IPs: http://jodies.de/ipcalc?host=10.30.1.192&mask1=32&mask2=

[angel0d]

just an update: with public IPs galera install proceeds and finishes fine. I am missing one IP though in this spare machine for c1rweb so as to fully test this

tried the standalone in-vps creation. it seems to install boa tools and populate a .barracuda.cnf. when running boa in-stable though in the newly created vps I noticed that it installs all PHP version. is this by design ?

other than that installation went flawlessly

great tools !!!

[omega8cc]

@angel0d Thanks for testing this and for the feedback!

We should test the installer with private IPs, obviously.

Yes, it is by design to install all PHP versions, so you don't need to edit .barracuda.cnf and run another upgrade, nor to use more complicated options on command line.

We should remove 5.5 from this list, though, but installing both 5.6 and 7.0 is probably a good idea, but suggestions on improvements of default configuration are always welcome!

[EdNett]

Hello, Is this supposed to be working currently in head? I tried just the first part:

Example for installing Linux VServer based BOA on a dedicated machine without any cluster configuration, just a BOA VPS:

with one IPv4 - is that all that is required, one IPv4 ? (I imagine IPv6's don't work, right?)

It didn't install anything at all, just downloaded a few vserver scripts, but didn't install anything, much less barracua or octopus. I don't want to issue a full error report - as nothing happened. I just want to know are people successfully using this?

Thank you.

ed

[omega8cc]

Yes, cluster support works in head, although there are still missing (important) parts, including running upgrades in the correct order, plus handling machine reboot situation, so the guests don't get stuck, etc. The single VPS mode should work fine, but needs more testing, so people don't get into trouble due to some edge case (machine/vendor/etc) situations.

[omega8cc]

That said, bug reports related to features introduced in head, so not officially supported yet, are more than welcome!

[angel0d]

regarding the in-vps installer it needs to improve the logic around the use of VMFAMILY internal variable when it equals to 'VS'

while searching for VMFAMILY there were 20+ files/scripts returned - maybe the many checks can remain but find another way to initialize VMFAMILY in a way that future install with in-vps are not considered as 'VS'

[angel0d]

as of January 29th the installer would fail during the in-all install step when used like

$ cluster in-host server.example.com $ reboot $ cluster up-host $ cluster in-all stable

with a lot of messages like ERROR 1045 (28000): ProxySQL Error: Access denied for user 'admin'@'' (using password: YES)

note in-all install step would fail with stable or head as install option

[angel0d]

Not sure if it should be a separate issue. I think csf is not properly install/working or I might be confusing something.

iptables and csf are install only on host system right ?

iptables --list returns an empty list of rules though

[omega8cc]

@angel0d Please open a separate issue for these problems, since there will be more of this kind to fix, I suspect.