bybai
commented
7 years ago This is my steps command list, it is not official doc now, will start official doc based on these content in next plan.
Take nfs based shared data as an example
nfs server:c910f05c01bc06 10.5.106.1 primary mn: bybc0609 10.5.106.9 secondary mn: bybc0605 . 10.5.106.5 use bybc0607 as test node, cn or sn: bybc0607 10.5.106.7 virtual ip address: 10.5.106.100 virtual hostname: byrhmn
configure primary and secondary xcat mn
on nfs server, export /HA:
[root@c910f05c01bc06 /]# cat /etc/exports|grep HA /HA (rw,no_root_squash,sync,no_subtree_check) [root@c910f05c01bc06 /]# export -a [root@c910f05c01bc06 /]# service nfs restart Redirecting to /bin/systemctl restart nfs.service [root@c910f05c01bc06 /]# showmount -e Export list for c910f05c01bc06: /HA [root@c910f05c01bc06 /]# mkdir /HA/etc/xcat [root@c910f05c01bc06 /]# mkdir -p /HA/root/.xcat [root@c910f05c01bc06 /]# mkdir -p /HA/install [root@c910f05c01bc06 /]# mkdir -p /HA/var/lib/pgsql [root@c910f05c01bc06 /]# mkdir -p /HA/tftpboot
on primary mn:
1, configure shared data: [root@bybc0609 ~]# mkdir /etc/xcat [root@bybc0609 ~]# mkdir /install/ [root@bybc0609 ~]# mkdir ~/.xcat [root@bybc0609 ~]# mkdir /var/lib/pgsql [root@bybc0609 ~]# mount -o rw c910f05c01bc06:/HA/etc/xcat /etc/xcat [root@bybc0609 ~]# mount -o rw c910f05c01bc06:/HA/root/.xcat ~/.xcat [root@bybc0609 ~]# mount -o rw c910f05c01bc06:/HA/install /install [root@bybc0609 ~]# mount -o rw c910f05c01bc06:/HA/var/lib/pgsql /var/lib/pgsql [root@bybc0609 data]# mount -o rw 10.5.106.1:/HA/tftpboot /tftpboot
[root@bybc0609 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/system-root 18G 981M 17G 6% / devtmpfs 885M 0 885M 0% /dev tmpfs 896M 0 896M 0% /dev/shm tmpfs 896M 8.5M 888M 1% /run tmpfs 896M 0 896M 0% /sys/fs/cgroup /dev/sda1 509M 137M 373M 27% /boot tmpfs 180M 0 180M 0% /run/user/0 c910f05c01bc06:/HA/etc/xcat 256G 233G 24G 91% /etc/xcat c910f05c01bc06:/HA/root/.xcat 256G 233G 24G 91% /root/.xcat c910f05c01bc06:/HA/install 256G 233G 24G 91% /install c910f05c01bc06:/HA/var/lib/pgsql 256G 233G 24G 91% /var/lib/pgsql c910f05c01bc06:/HA/tftpboot 256G 238G 18G 91% /tftpboot
2, install xCAT based on xCAT doc, add the two management nodes into policy table: [root@bybc0609 x86_64]# tabedit policy "1.2","bybc0609.cluster.com",,,,,,"trusted",, "1.3","bybc0605.cluster.com",,,,,,"trusted",, "1.4","byrhmn.cluster.com",,,,,,"trusted",,
3, switch database to postgresql [root@bybc0609 x86_64]# chdef -t site databaseloc=/var/lib/pgsql [root@bybc0609 x86_64]# yum -y install postgresql* [root@bybc0609 x86_64]# yum -y install perl-DBD-Pg.x86_64 [root@bybc0609 x86_64]# pgsqlsetup -i -V
4, set up virtual ip address on primary mn, configure virtual ip address in /etc/hosts /etc/resolv.conf on primary mn, update /etc/nsswitch.conf [root@bybc0609 ~]# ifconfig eth0:0 10.5.106.100 netmask 255.0.0.0 [root@bybc0609 ~]# ip address show eth0 |grep inet inet 10.5.106.9/8 brd 10.255.255.255 scope global dynamic eth0 inet 10.5.106.100/8 brd 10.255.255.255 scope global secondary eth0:0 [root@bybc0609 ~]# cat /etc/resolv.conf search cluster.com. nameserver 10.5.106.100 [root@bybc0609 ~]# cat /etc/hosts|grep 09 10.5.106.100 bybc0609 bybc0609.cluster.com [root@bybc0609 ~]# grep hosts /etc/nsswitch.conf hosts: files dns myhostname
5, let xcat use virtual ip, change site table attribute master,nameserver,tftpserver etc, use following command to check; lsdef -t site -l
6, configure xcatdb use postgresql. add virtual ip in postgresql configure files pg_hba.conf and postgresql.conf , restart postgresql service, restart xcatd service. [root@bybc0609 data]# cat /var/lib/pgsql/data/pg_hba.conf | grep host host all all 10.5.106.9/32 md5 host all all 10.5.106.100/32 md5 host all all 10.5.106.5/32 md5 host all all 10.5.106.7/32 md5 host all all 127.0.0.1/32 trust host all all ::1/128 trust [root@bybc0609 data]# cat /var/lib/pgsql/data/postgresql.conf |grep listen listen_addresses = 'localhost,10.5.106.9,10.5.106.100,10.5.106.5'
7, restart db and xcatd service [root@bybc0609 ~]# hostname byrhmn [root@bybc0609 data]#service postgresql restart [root@bybc0609 data]#service xcatd restart
8, check db, for example, node definition, replace all 10.5.106.9 to 10.5.106.100,
9, stop xcatd and db service, in order to setup standby MN [root@bybc0609 ~]# service xcatd stop Stopping xcatd (via systemctl): [ OK ] [root@bybc0609 ~]# service postgresql stop Redirecting to /bin/systemctl stop postgresql.service [root@bybc0609 ~]# ifconfig eth0:0 0.0.0.0 0.0.0.0
on Standby MN node:
1, install xcat using ip 10.5.106.5 2, add virtual ip 10.5.106.100 [root@bybc0605 x86_64]# ifconfig eth0:0 10.5.106.100 netmask 255.0.0.0 [root@bybc0605 x86_64]# grep 10.5.106.100 /etc/resolv.conf nameserver 10.5.106.100 [root@bybc0605 x86_64]# grep 10.5.106.100 /etc/hosts 10.5.106.100 bybc0605 bybc0605.cluster.com [root@bybc0605 x86_64]# grep hosts /etc/nsswitch.conf hosts: files dns myhostname
3, change xcat use virtual ip 10.5.106.100, check site table attribute, then restart xcatd [root@bybc0605 x86_64]# lsdef -t site -l [root@bybc0605 x86_64]# service xcatd restart
4, Setup ssh authentication between the primary management node and standby management node. It should be setup as “passwordless ssh authentication” and it should work in both directions. The summary of this procedure is:
- cat keys from /.ssh/id_rsa.pub on the primary management node and add them to /.ssh/authorized_keys on the standby management node. Remove the standby management node entry from /.ssh/known_hosts on the primary management node prior to issuing ssh to the standby management node.
- cat keys from /.ssh/id_rsa.pub on the standby management node and add them to /.ssh/authorized_keys on the primary management node. Remove the primary management node entry from /.ssh/known_hosts on the standby management node prior to issuing ssh to the primary management node.
5, Make sure the time on the primary management node and standby management node is synchronized.
6, install postgresql: [root@bybc0605 x86_64]# chdef -t site databaseloc=/var/lib/pgsql [root@bybc0605 x86_64]# yum -y install postgresql* [root@bybc0605 x86_64]# yum -y install perl-DBD-Pg.x86_64
7, stop xcatd [root@bybc0605 .ssh]# service xcatd stop [root@bybc0605 .ssh]# ifconfig eth0:0 0.0.0.0 0.0.0.0
8, back to primary mn, start postgresql and xcatd, using primary xcat MN
rsync ssh keys and /etc/hosts file Add the following in current primary MN crontab: 0 1 /usr/bin/rsync -Lprgotz $HOME/.ssh/id bybc0605:$HOME/.ssh/ 0 2 * /usr/bin/rsync -Lprogtz /etc/hosts bybc0605:/etc/
Failover
On the current primary management node:
1, Stop the xCAT daemon [root@bybc0609 ~]# service xcatd stop Stopping xcatd (via systemctl): [ OK ] [root@bybc0609 ~]# service dhcpd stop [root@bybc0609 ~]# service postgresql stop Redirecting to /bin/systemctl stop postgresql.service
2, unexport the xCAT NFS directories [root@bybc0609 ~]# exportfs -ua
3, Unmount shared data [root@bybc0609 ~]# umount /etc/xcat [root@bybc0609 ~]# umount /install [root@bybc0609 ~]# umount ~/.xcat [root@bybc0609 ~]# umount /tftpboot [root@bybc0609 ~]# umount /var/lib/pgsql
4, unconfigure virtual ip [root@bybc0609 ~]# ifconfig eth0:0 0.0.0.0 0.0.0.0
on new primary mn(original standby mn)：
1, Configure Virtual IP: [root@bybc0605 x86_64]# ifconfig eth0:0 10.5.106.100 netmask 255.0.0.0 [root@bybc0605 ~]# hostname byrhmn
2, mount shared data: [root@byrhmn ~]# mount -o rw c910f05c01bc06:/HA/etc/xcat /etc/xcat [root@byrhmn ~]# mount -o rw c910f05c01bc06:/HA/root/.xcat ~/.xcat [root@byrhmn ~]# mount -o rw c910f05c01bc06:/HA/install /install [root@byrhmn ~]# mount -o rw c910f05c01bc06:/HA/var/lib/pgsql /var/lib/pgsql [root@byrhmn ~]# mount -o rw 10.5.106.1:/HA/tftpboot /tftpboot [root@byrhmn ~]#mkdir -p /install/netboot/rhels7.4/x86_64/compute [root@byrhmn ~]#mkdir -p /tmp/rootimg [root@byrhmn ~]#ln -s /tmp/rootimg /install/netboot/rhels7.4/x86_64/compute
3, start postgresql, xcatd, dhcpd etc [root@byrhmn ~]# service postgresql start [root@byrhmn ~]# service xcatd start [root@byrhmn ~]#makedns -n [root@byrhmn ~]#makedhcp -n [root@byrhmn ~]#makedhcp -a
Verification:
1. provision diskless cn directly
copycds RHEL-7.4-20170711.0-Server-x86_64-dvd1.iso genimage rhels7.4-x86_64-netboot-compute packimage rhels7.4-x86_64-netboot-compute cp /tmp/rootimg.cpio.gz /install/netboot/rhels7.4/x86_64/compute/ chmod 644 /install/netboot/rhels7.4/x86_64/compute/rootimg.cpio.gz cat bybc0607 |chdef -z makehosts bybc0607 makedns -n makedhcp -n makedhcp -a nodeset bybc0607 osimage=rhels7.4-x86_64-netboot-compute rpower bybc0607 reset
2. provision sn
[root@byrhmn xcat]# chtab key=nameservers site.value="
This task is in order to save setup xcat MN HA using shared data steps command list.