Beaker snippets not being pulled during provisioning

[shall1024]

@dledford reports that RHEL provisions are not pulling the snippets we have defined in /etc/beaker/snippets, and is instead using the default Beaker snippets.

Currently, we are running provisions to see if the issue is consistent across all users or not; in my provisions, Beaker is at least using the snippets during installation but I will need to check if the kickstart file saved on the provisioned system reflects this.

We will both update with the contents of our files and, from there, I will decide where to start solving this problem.

[shall1024]

I ran a provision of RHEL 7.9 on node-06, which generated the following anaconda-ks.cfg:

#version=DEVEL
# Use text mode install
text
# Firewall configuration
firewall --disabled
firstboot --disable
ignoredisk --only-use=sda
# Keyboard layouts
# old format: keyboard us
# new format:
keyboard --vckeymap=us --xlayouts=''
# System language
lang en_US.UTF-8

# Network information
network  --bootproto=dhcp --hostname=node-06.ofa.iol.unh.edu --activate
# Reboot after installation
reboot
repo --name="beaker-Server-HighAvailability" --baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/addons/HighAvailability --cost=100
repo --name="beaker-Server-ResilientStorage" --baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/addons/ResilientStorage --cost=100
repo --name="beaker-Server" --baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/ --cost=100
repo --name="Server-HighAvailability" --baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd//addons/HighAvailability
repo --name="Server-ResilientStorage" --baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd//addons/ResilientStorage
# Use network installation
url --url="http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/"
# Root password
rootpw --iscrypted $1$yocb6JIG$ehIHzyAnTlqiCfm.jDx3f/
# SELinux configuration
selinux --enforcing
# System services
services --enabled="chronyd"
# Do not configure the X Window System
skipx
# System timezone
timezone America/New_York --ntpservers=gateway
# System bootloader configuration
bootloader --append="console=ttyS0,115200n8 crashkernel=auto" --location=mbr --boot-drive=sda
autopart --type=lvm
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel

%pre --logfile=/dev/console
set -x
# Some distros have curl in their minimal install set, others have wget.
# We define a wrapper function around the best available implementation
# so that the rest of the script can use that for making HTTP requests.
if command -v curl >/dev/null ; then
    # Older curl versions lack --retry
    if curl --help 2>&1 | grep -q .*--retry ; then
        function fetch() {
            curl -L --retry 20 --remote-time -o "$1" "$2"
        }
    else
        function fetch() {
            curl -L --remote-time -o "$1" "$2"
        }
    fi
elif command -v wget >/dev/null ; then
    # In Anaconda images wget is actually busybox
    if wget --help 2>&1 | grep -q BusyBox ; then
        function fetch() {
            wget -O "$1" "$2"
        }
    else
        function fetch() {
            wget --tries 20 -O "$1" "$2"
        }
    fi
else
    echo "No HTTP client command available!"
    function fetch() {
        false
    }
fi

# Check in with Beaker Server
fetch - http://beaker.ofa.iol.unh.edu:8000/install_start/278

if command -v python3 >/dev/null ; then
   fetch /tmp/anamon http://beaker.ofa.iol.unh.edu/beaker/anamon3
   python_command="python3"
elif [ -f /usr/libexec/platform-python ] && \
     /usr/libexec/platform-python --version 2>&1 | grep -q "Python 3" ; then
   fetch /tmp/anamon http://beaker.ofa.iol.unh.edu/beaker/anamon3
   python_command="/usr/libexec/platform-python"
else
   fetch /tmp/anamon http://beaker.ofa.iol.unh.edu/beaker/anamon
   python_command="python"
fi
$python_command /tmp/anamon --recipe-id 278 --xmlrpc-url 'http://beaker.ofa.iol.unh.edu:8000/RPC2'
# no snippet data for RedHatEnterpriseLinux7_pre
# no snippet data for RedHatEnterpriseLinux_pre
%end

%post --logfile=/dev/console
set -x
# Some distros have curl in their minimal install set, others have wget.
# We define a wrapper function around the best available implementation
# so that the rest of the script can use that for making HTTP requests.
if command -v curl >/dev/null ; then
    # Older curl versions lack --retry
    if curl --help 2>&1 | grep -q .*--retry ; then
        function fetch() {
            curl -L --retry 20 --remote-time -o "$1" "$2"
        }
    else
        function fetch() {
            curl -L --remote-time -o "$1" "$2"
        }
    fi
elif command -v wget >/dev/null ; then
    # In Anaconda images wget is actually busybox
    if wget --help 2>&1 | grep -q BusyBox ; then
        function fetch() {
            wget -O "$1" "$2"
        }
    else
        function fetch() {
            wget --tries 20 -O "$1" "$2"
        }
    fi
else
    echo "No HTTP client command available!"
    function fetch() {
        false
    }
fi

# Check in with Beaker Server, let it know our hostname, and
# record our install time.

# We will try a number of different places to figure out the system's FQDN.
# In all cases we will only accept a real FQDN (no "localhost", and must have
# a domain portion). DNS is our preferred source, otherwise the installer
# should have stored a hostname in /etc based on the kickstart or DHCP info.
# As a last resort we will use the system's first IP address.
function find_fqdn() {
    local fqdn=
    # hostname -f is the most future-proof approach, but it isn't always reliable
    fqdn=$(hostname -f)
    if [[ "$fqdn" == *.* && "$fqdn" != localhost.* ]] ; then echo "$fqdn" ; return ; fi
    # Preferred fallback if the OS is recent enough to provide it
    fqdn=$(cat /etc/hostname)
    if [[ "$fqdn" == *.* && "$fqdn" != localhost.* ]] ; then echo "$fqdn" ; return ; fi
    # Red Hat-based systems prior to systemd will have this
    fqdn=$(grep ^HOSTNAME= /etc/sysconfig/network | cut -f2- -d=)
    if [[ "$fqdn" == *.* && "$fqdn" != localhost.* ]] ; then echo "$fqdn" ; return ; fi
    # Getting desperate... pick the first local IP address
    ipaddr=$(hostname -i)
    if [[ "$ipaddr" != "127.0.0.1" ]] ; then echo "$ipaddr" ; return ; fi
    # Getting even more desperate (RHEL5 and earlier)
    ip addr show | grep -v ' lo' | grep -Po '(?<=inet )[0-9.]+'
}
REPORTED_FQDN=$(find_fqdn)
fetch - "http://beaker.ofa.iol.unh.edu:8000/install_done/278/$REPORTED_FQDN"
fetch - http://beaker.ofa.iol.unh.edu:8000/nopxe/node-06.ofa.iol.unh.edu
echo 278 > /root/RECIPE.TXT

# If netboot_method= is found in /proc/cmdline record it to /root
netboot_method=$(grep -oP "(?<=netboot_method=)[^\s]+(?=)" /proc/cmdline)
if [ -n "$netboot_method" ]; then
echo $netboot_method >/root/NETBOOT_METHOD.TXT
fi

# Enable post-install boot notification

if command -v python3 >/dev/null ; then
    fetch /usr/local/sbin/anamon http://beaker.ofa.iol.unh.edu/beaker/anamon3
elif [ -f /usr/libexec/platform-python ] && \
     /usr/libexec/platform-python --version 2>&1 | grep -q "Python 3" ; then
    fetch /usr/local/sbin/anamon http://beaker.ofa.iol.unh.edu/beaker/anamon3
    sed -i 's/#!\/usr\/bin\/python3/#!\/usr\/libexec\/platform-python/' /usr/local/sbin/anamon
else
    fetch /usr/local/sbin/anamon http://beaker.ofa.iol.unh.edu/beaker/anamon
fi
chmod 755 /usr/local/sbin/anamon

# OS without `initscripts` need to use systemd
if [ -e /etc/init.d/functions ]; then
    fetch /etc/rc.d/init.d/anamon http://beaker.ofa.iol.unh.edu/beaker/anamon.init

    chmod 755 /etc/rc.d/init.d/anamon
    if selinuxenabled &>/dev/null ; then
        restorecon /etc/rc.d/init.d/anamon /usr/local/sbin/anamon
    fi

    chkconfig --add anamon
else
    fetch /etc/systemd/system/anamon.service http://beaker.ofa.iol.unh.edu/beaker/anamon.service
    systemctl enable anamon
fi

cat << __EOT__ > /etc/sysconfig/anamon
XMLRPC_URL="http://beaker.ofa.iol.unh.edu:8000/RPC2"
RECIPE_ID="278"
LOGFILES="/var/log/boot.log /var/log/messages /var/log/dmesg"
__EOT__

if [ -f /etc/sysconfig/readahead ] ; then
    :
    cat >>/etc/sysconfig/readahead <<EOF

# readahead conflicts with auditd, see bug 561486 for detailed explanation.
#
# Should a task need to change these settings, it must revert to this state
# when test is done.

READAHEAD_COLLECT="no"
READAHEAD_COLLECT_ON_RPM="no"
EOF
fi
systemctl disable systemd-readahead-collect.service

if [ -e /etc/sysconfig/ntpdate ] ; then
    systemctl enable ntpdate.service
fi
if [ -e "/etc/sysconfig/ntpd" ]; then
    systemctl enable ntpd.service
    GOT_G=$(/bin/cat /etc/sysconfig/ntpd | grep -E '^OPTIONS' | grep '\-g')

    if [ -z "$GOT_G" ]; then
        /bin/sed -i -r 's/(^OPTIONS\s*=\s*)(['\''|"])(.+)$/\1\2\-x \3 /' /etc/sysconfig/ntpd
    fi
fi
if [ -e /etc/chrony.conf ] ; then
    cp /etc/chrony.conf{,.orig}
    # use only DHCP-provided time servers, no default pool servers
    sed -i '/^server /d' /etc/chrony.conf
    cp /etc/sysconfig/network{,.orig}
    # setting iburst should speed up initial sync
    # https://bugzilla.redhat.com/show_bug.cgi?id=787042#c12
    echo NTPSERVERARGS=iburst >>/etc/sysconfig/network
    systemctl disable ntpd.service
    systemctl disable ntpdate.service
    systemctl enable chronyd.service
    systemctl enable chrony-wait.service
fi

if efibootmgr &>/dev/null ; then
    # grab the PXE IPv4 FlexibleLOM from the boot order
    bootnum=$(efibootmgr | grep "Embedded FlexibleLOM 1 Port 1 : HPE Ethernet 1Gb 4-port 366FLR Adapter - NIC (PXE IPv4)")  
    # trim the actual boot order number from the value
    bootnum=${bootnum:4:5}
    bootnum=${bootnum:0:4}

    # extract the bootnum out of the boot order and put it at the front:
    boot_order=$(efibootmgr | awk '/BootOrder/ { print $2 }')
    boot_order_test=${boot_order//$bootnum,}
    #if boot device is at the end of the list,
    if [ "$boot_order_test" = "$boot_order" ]; then
        new_boot_order=${boot_order//$bootnum}
        new_boot_order=${bootnum},${new_boot_order}
        new_boot_order=${new_boot_order:: -1}
    else # device is in middle or beginning of the list.
        new_boot_order=${boot_order//$bootnum,}
        new_boot_order=${bootnum},${new_boot_order}
    fi
    # Change the boot order to boot from network boot.
    efibootmgr -o "$new_boot_order"
fi

#Add Task Repo
cat <<"EOF" >/etc/yum.repos.d/beaker-tasks.repo
[beaker-tasks]
name=beaker-tasks
baseurl=http://beaker.ofa.iol.unh.edu/repos/278
enabled=1
gpgcheck=0
skip_if_unavailable=0
EOF

# Add Harness Repo
cat <<"EOF" >/etc/yum.repos.d/beaker-harness.repo
[beaker-harness]
name=beaker-harness
baseurl=http://beaker.ofa.iol.unh.edu/harness/RedHatEnterpriseLinux7/
enabled=1
gpgcheck=0
EOF

# Add distro and custom Repos
cat <<"EOF" >/etc/yum.repos.d/beaker-Server-HighAvailability.repo
[beaker-Server-HighAvailability]
name=beaker-Server-HighAvailability
baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/addons/HighAvailability
enabled=1
gpgcheck=0
skip_if_unavailable=1
EOF
cat <<"EOF" >/etc/yum.repos.d/beaker-Server-ResilientStorage.repo
[beaker-Server-ResilientStorage]
name=beaker-Server-ResilientStorage
baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/addons/ResilientStorage
enabled=1
gpgcheck=0
skip_if_unavailable=1
EOF
cat <<"EOF" >/etc/yum.repos.d/beaker-Server.repo
[beaker-Server]
name=beaker-Server
baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/
enabled=1
gpgcheck=0
skip_if_unavailable=1
EOF

if command -v dnf >/dev/null ; then
   package_command="dnf"
else
   package_command="yum"
fi
# fill the yum cache and redirect output to /dev/null
# This speeds up yum because of a bug where it will update stdout too often.
# http://lists.baseurl.org/pipermail/yum-devel/2011-December/008857.html
$package_command check-update -y > /dev/null 2>&1 || true
cat <<"EOF" >/etc/profile.d/beaker-harness-env.sh
export BEAKER_LAB_CONTROLLER_URL="http://beaker.ofa.iol.unh.edu:8000/"
export BEAKER_LAB_CONTROLLER=beaker.ofa.iol.unh.edu
export BEAKER_RECIPE_ID=278
export BEAKER_HUB_URL="http://beaker.ofa.iol.unh.edu/bkr/"
EOF
cat <<"EOF" >/etc/profile.d/beaker-harness-env.csh
setenv BEAKER_LAB_CONTROLLER_URL "http://beaker.ofa.iol.unh.edu:8000/"
setenv BEAKER_LAB_CONTROLLER beaker.ofa.iol.unh.edu
setenv BEAKER_RECIPE_ID 278
setenv BEAKER_HUB_URL "http://beaker.ofa.iol.unh.edu/bkr/"
EOF

if command -v dnf >/dev/null ; then
   package_command="dnf"
else
   package_command="yum"
fi
$package_command -y install restraint-rhts

#Add test user account
useradd --password '$6$oIW3o2Mr$XbWZKaM7nA.cQqudfDJScupXOia5h1u517t6Htx/Q/MgXm82Pc/OcytatTeI4ULNWOMJzvpCigWiL4xKP9PX4.' test
cat <<"EOF" >/etc/profile.d/beaker.sh
export BEAKER="http://beaker.ofa.iol.unh.edu/bkr/"
export BEAKER_JOB_WHITEBOARD='Reserve Workflow provision of distro Red Hat Enterprise Linux 7.9 on a specific system for 1 hour'
export BEAKER_RECIPE_WHITEBOARD=''
EOF
cat <<"EOF" >/etc/profile.d/beaker.csh
setenv BEAKER "http://beaker.ofa.iol.unh.edu/bkr/"
setenv BEAKER_JOB_WHITEBOARD 'Reserve Workflow provision of distro Red Hat Enterprise Linux 7.9 on a specific system for 1 hour'
setenv BEAKER_RECIPE_WHITEBOARD ''
EOF
cat << EOF > /etc/profile.d/rh-env.sh
export LAB_CONTROLLER=beaker.ofa.iol.unh.edu
export DUMPSERVER=""

# Inside Red Hat, the NFSSERVERS variable points to mounts for each of the RHEL
# releases supported, so for example
# rhel5-nfs.blah.foo.bar.redhat.com:/export/home which would then have a
# a directory structure of <fqdn_of_test_machine>/ and in that directory
# would be freeform files that can be used in a test setting.
export NFSSERVERS=""

# The lookaside area is for non-release specific files to be downloaded.
# Generally, they are user space programs, scripts, or tests that are
# expected to work on any install and don't need a per-release area.
# All of the rdma-setup scripts are in this area inside the Red Hat cluster
export LOOKASIDE=http://builder-01.ofa.iol.unh.edu/lookaside

# This is the root directory for downloading build files.  This is not
# like something we'll need inside the OFA cluster as we don't have internal,
# private builds like we do inside Red Hat
export BUILDURL=http://download.eng.bos.redhat.com
EOF
mkdir -p /root/.ssh
cat >>/root/.ssh/authorized_keys <<"__EOF__"
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFePyDLPHQVSAxoET1gMpc6hIJ/lOPpSI8oJBtvkMJP5HH6SdmA6YWmyFyoO3MMlznAfs+4/7O5Zz3mPNIE21tOmFCH7ESEpR7fYnpkFXa9bVnFONJxFAlmJXhdQIqpMr1SR+h2b2LcGbYIAF6N6mnl+nJccoHnbUqIMafnMR0f54sN942MsPQ7tkmxQVty/8JaXiFfvappSjdFCYX8CyKtC3XXkVFMrAVrTn3Gu2Fwb415bmkIQ+bfzz1npcKz0Kg6Z3qWq5t8ALm/AFRGj+z1ln0kjRT5TNdxkeotNLdbpgouZhZMYu6/kAKSn0oCkUtUA++KFEG1OruAtonfuT3MtBykLaQq/KGfxDZWuclq9ciz+s/cnMuGXoCmsRcUv1GJ5mIxvJ9INg/3wJAyCXDOrThG3J+HTGJpdNwbOSKg4xMVhd2cqrVGGFyOpB0sur2xRy76rLmxHItQTPMnFsiYejAdJX7kKkzs0Nk95k4FSQpQPq9qekuEMXRP1QAlhk= shall@iol.unh.edu
__EOF__
restorecon -R /root/.ssh
chmod go-w /root /root/.ssh /root/.ssh/authorized_keys
# Disable rhts-compat for Fedora15/RHEL7 and newer.
cat >> /etc/profile.d/task-overrides-rhts.sh <<END
export RHTS_OPTION_COMPATIBLE=
export RHTS_OPTION_COMPAT_SERVICE=
END
# no snippet data for RedHatEnterpriseLinux7_post
# no snippet data for RedHatEnterpriseLinux_post
%end

%onerror
set -x
# Some distros have curl in their minimal install set, others have wget.
# We define a wrapper function around the best available implementation
# so that the rest of the script can use that for making HTTP requests.
if command -v curl >/dev/null ; then
    # Older curl versions lack --retry
    if curl --help 2>&1 | grep -q .*--retry ; then
        function fetch() {
            curl -L --retry 20 --remote-time -o "$1" "$2"
        }
    else
        function fetch() {
            curl -L --remote-time -o "$1" "$2"
        }
    fi
elif command -v wget >/dev/null ; then
    # In Anaconda images wget is actually busybox
    if wget --help 2>&1 | grep -q BusyBox ; then
        function fetch() {
            wget -O "$1" "$2"
        }
    else
        function fetch() {
            wget --tries 20 -O "$1" "$2"
        }
    fi
else
    echo "No HTTP client command available!"
    function fetch() {
        false
    }
fi
fetch - http://beaker.ofa.iol.unh.edu:8000/install_fail/278
sleep 10
%end

%post
set -x
# Some distros have curl in their minimal install set, others have wget.
# We define a wrapper function around the best available implementation
# so that the rest of the script can use that for making HTTP requests.
if command -v curl >/dev/null ; then
    # Older curl versions lack --retry
    if curl --help 2>&1 | grep -q .*--retry ; then
        function fetch() {
            curl -L --retry 20 --remote-time -o "$1" "$2"
        }
    else
        function fetch() {
            curl -L --remote-time -o "$1" "$2"
        }
    fi
elif command -v wget >/dev/null ; then
    # In Anaconda images wget is actually busybox
    if wget --help 2>&1 | grep -q BusyBox ; then
        function fetch() {
            wget -O "$1" "$2"
        }
    else
        function fetch() {
            wget --tries 20 -O "$1" "$2"
        }
    fi
else
    echo "No HTTP client command available!"
    function fetch() {
        false
    }
fi
# Check in with Beaker Server, record our postinstall time.
fetch - http://beaker.ofa.iol.unh.edu:8000/postinstall_done/278
# Give anamon a chance to finish collecting logs.
sleep 10
%end

%packages --ignoremissing
atop
bash-completion
bonnie++
chrony
dbench
ftop
git-all
htop
ibutils
iftop
infiniband-diags
iotop
iperf3
iscsi-initiator-utils
kexec-tools
latencytop-tui
libcxgb4
libhfi1
libibverbs-utils
libmlx4
libmlx5
libpsm2
librdmacm-utils
netperf
nfs-utils
ntop
numatop
nuttcp
opa-address-resolution
opa-basic-tools
opa-ff
pciutils
perf
perftest
qperf
sockperf
srptools
tiotest
tiptop
uperf
vim-X11
vim-enhanced

%end

%addon com_redhat_kdump --enable --reserve-mb='auto'

%end

Upon checking the system, and the installation logs, it seems like all of these packages installed correctly. I am going to check if there is some sort of permissions issue, as that's what I suspect. Please let me know what your output looks like, or if this output is unexpected. Thank you.

[dledford]

I think we need to be clear about how we are installing the machines. I've been manually provisioning the machines, so that means I have to loan the machine to myself (I shouldn't have to do this step, but I do for some reason), then I can take the machine, then I click on the Provision tab and I manually provision the machine. The other way is to do an automated provision, and I think that's what you might be using @shall1024. One of the significant differences between the two is that you have the test harness setup, whereas my machine does not.

When I provision a machine, not all of the snippets are applied. Some are though. From what I can tell, the Anaconda kickstart used during the install is saved in /root as original-ks.cfg. I'm not entirely sure what the anaconda-ks.cfg file is as it doesn't appear to be used. But, doing a diff between the two clearly shows that the original-ks.cfg has the package list snippet intact. I can run htop from the command line after the install, and htop is one of the packages in the original-ks.cfg package list, so that verifies that the original-ks.cfg was used during the install. However, the system_post snippet is not being applied and doesn't appear to be applied in your kickstart either.

I'm working on some updates to the snippets. The git-all package doesn't exist on all distros, so we need to pull in the regular git package instead. And we need to get that system_post snippet working. Also, the epel repo definition needs --nogpgcheck instead of --nosslverify (or maybe in addition to --nosslverify, I'm not sure about that one).

Here's my original-ks.cfg for reference:

url --url=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/

text
# System bootloader configuration
bootloader --location=mbr --append="console=ttyS0,115200n8"
network --bootproto=dhcp --hostname=node-01.ofa.iol.unh.edu

repo --name=beaker-Server-HighAvailability --cost=100 --baseurl=http://beaker.ofa.iol.unh.edu/distro
s/rhel-server-7.9-x86_64-dvd/addons/HighAvailability
repo --name=beaker-Server-ResilientStorage --cost=100 --baseurl=http://beaker.ofa.iol.unh.edu/distro
s/rhel-server-7.9-x86_64-dvd/addons/ResilientStorage
repo --name=beaker-Server --cost=100 --baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9
-x86_64-dvd/

firewall --disabled
# Run the Setup Agent on first boot
firstboot --disable
# System keyboard
keyboard us
# System language
lang en_US.UTF-8

reboot
#Root password
rootpw --iscrypted $1$3hWp58hF$ChUGUPzfJhau7XF89QT7e0
# SELinux configuration
selinux --enforcing

timezone America/New_York

zerombr
clearpart --all --initlabel

autopart
# no snippet data for RedHatEnterpriseLinux7
# no snippet data for RedHatEnterpriseLinux
repo --name=epel --mirrorlist=https://mirrors.fedoraproject.org/mirrorlist?repo=epel-7&arch=x86_64 -
-install --noverifyssl

%packages --ignoremissing --default
# Generally useful OS tools
pciutils
nfs-utils
iscsi-initiator-utils
git-all
bash-completion
vim-X11
vim-enhanced

# Overall RDMA tools
libibverbs-utils
librdmacm-utils
infiniband-diags
ibutils
srptools

# Driver specific packages
libmlx4
libmlx5
libcxgb4
libhfi1
opa-ff
opa-address-resolution
opa-basic-tools
libpsm2

# Performance testers for RDMA
qperf
perftest

# Additional packages...not in base RHEL, requires EPEL to be enabled
perf
iperf3
uperf
sockperf
nuttcp
netperf
atop
ftop
htop
iftop
iotop
latencytop-tui
ntop
numatop
tiptop
bonnie++
dbench
tiotest
chrony
%end

%pre --log=/dev/console
set -x
# Some distros have curl in their minimal install set, others have wget.
# We define a wrapper function around the best available implementation
# so that the rest of the script can use that for making HTTP requests.
if command -v curl >/dev/null ; then
    # Older curl versions lack --retry
    if curl --help 2>&1 | grep -q .*--retry ; then
        function fetch() {
            curl -L --retry 20 --remote-time -o "$1" "$2"
        }
    else
        function fetch() {
            curl -L --remote-time -o "$1" "$2"
        }
    fi
elif command -v wget >/dev/null ; then
    # In Anaconda images wget is actually busybox
    if wget --help 2>&1 | grep -q BusyBox ; then
        function fetch() {
            wget -O "$1" "$2"
        }
    else
        function fetch() {
            wget --tries 20 -O "$1" "$2"
        }
    fi
else
    echo "No HTTP client command available!"
    function fetch() {
        false
    }
fi

# no snippet data for RedHatEnterpriseLinux7_pre
# no snippet data for RedHatEnterpriseLinux_pre
%end

%post --log=/dev/console
set -x
# Some distros have curl in their minimal install set, others have wget.
# We define a wrapper function around the best available implementation
# so that the rest of the script can use that for making HTTP requests.
if command -v curl >/dev/null ; then
    # Older curl versions lack --retry
    if curl --help 2>&1 | grep -q .*--retry ; then
        function fetch() {
            curl -L --retry 20 --remote-time -o "$1" "$2"
        }
    else
        function fetch() {
            curl -L --remote-time -o "$1" "$2"
        }
    fi
elif command -v wget >/dev/null ; then
    # In Anaconda images wget is actually busybox
    if wget --help 2>&1 | grep -q BusyBox ; then
        function fetch() {
            wget -O "$1" "$2"
        }
    else
        function fetch() {
            wget --tries 20 -O "$1" "$2"
        }
    fi
else
    echo "No HTTP client command available!"
    function fetch() {
        false
    }
fi

fetch - http://beaker.ofa.iol.unh.edu:8000/nopxe/node-01.ofa.iol.unh.edu

# If netboot_method= is found in /proc/cmdline record it to /root
netboot_method=$(grep -oP "(?<=netboot_method=)[^\s]+(?=)" /proc/cmdline)
if [ -n "$netboot_method" ]; then
echo $netboot_method >/root/NETBOOT_METHOD.TXT
fi

# Enable post-install boot notification
if [ -f /etc/sysconfig/readahead ] ; then
    :
    cat >>/etc/sysconfig/readahead <<EOF

# readahead conflicts with auditd, see bug 561486 for detailed explanation.
#
# Should a task need to change these settings, it must revert to this state
# when test is done.

READAHEAD_COLLECT="no"
READAHEAD_COLLECT_ON_RPM="no"
EOF
fi
systemctl disable systemd-readahead-collect.service

if [ -e /etc/sysconfig/ntpdate ] ; then
    systemctl enable ntpdate.service
fi
if [ -e "/etc/sysconfig/ntpd" ]; then
    systemctl enable ntpd.service
    GOT_G=$(/bin/cat /etc/sysconfig/ntpd | grep -E '^OPTIONS' | grep '\-g')

    if [ -z "$GOT_G" ]; then
        /bin/sed -i -r 's/(^OPTIONS\s*=\s*)(['\''|"])(.+)$/\1\2\-x \3 /' /etc/sysconfig/ntpd
    fi
fi
if [ -e /etc/chrony.conf ] ; then
    cp /etc/chrony.conf{,.orig}
    # use only DHCP-provided time servers, no default pool servers
    sed -i '/^server /d' /etc/chrony.conf
    cp /etc/sysconfig/network{,.orig}
    # setting iburst should speed up initial sync
    # https://bugzilla.redhat.com/show_bug.cgi?id=787042#c12
    echo NTPSERVERARGS=iburst >>/etc/sysconfig/network
    systemctl disable ntpd.service
    systemctl disable ntpdate.service
    systemctl enable chronyd.service
    systemctl enable chrony-wait.service
fi

if efibootmgr &>/dev/null ; then
    # grab the PXE IPv4 FlexibleLOM from the boot order
    bootnum=$(efibootmgr | grep "Embedded FlexibleLOM 1 Port 1 : HPE Ethernet 1Gb 4-port 366FLR Adap
ter - NIC (PXE IPv4)")
    # trim the actual boot order number from the value
    bootnum=${bootnum:4:5}
    bootnum=${bootnum:0:4}

    # extract the bootnum out of the boot order and put it at the front:
    boot_order=$(efibootmgr | awk '/BootOrder/ { print $2 }')
    boot_order_test=${boot_order//$bootnum,}
    #if boot device is at the end of the list,
    if [ "$boot_order_test" = "$boot_order" ]; then
        new_boot_order=${boot_order//$bootnum}
        new_boot_order=${bootnum},${new_boot_order}
        new_boot_order=${new_boot_order:: -1}
    else # device is in middle or beginning of the list.
        new_boot_order=${boot_order//$bootnum,}
        new_boot_order=${bootnum},${new_boot_order}
    fi
    # Change the boot order to boot from network boot.
    efibootmgr -o "$new_boot_order"
fi

# Add distro and custom Repos
cat <<"EOF" >/etc/yum.repos.d/beaker-Server-HighAvailability.repo
[beaker-Server-HighAvailability]
name=beaker-Server-HighAvailability
baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/addons/HighAvailability
enabled=1
gpgcheck=0
skip_if_unavailable=1
EOF
cat <<"EOF" >/etc/yum.repos.d/beaker-Server-ResilientStorage.repo
[beaker-Server-ResilientStorage]
name=beaker-Server-ResilientStorage
baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/addons/ResilientStorage
enabled=1
gpgcheck=0
skip_if_unavailable=1
EOF
cat <<"EOF" >/etc/yum.repos.d/beaker-Server.repo
[beaker-Server]
name=beaker-Server
baseurl=http://beaker.ofa.iol.unh.edu/distros/rhel-server-7.9-x86_64-dvd/
enabled=1
gpgcheck=0
skip_if_unavailable=1
EOF

#Add test user account
useradd --password '$6$oIW3o2Mr$XbWZKaM7nA.cQqudfDJScupXOia5h1u517t6Htx/Q/MgXm82Pc/OcytatTeI4ULNWOMJ
zvpCigWiL4xKP9PX4.' test
cat <<"EOF" >/etc/profile.d/beaker.sh
export BEAKER="http://beaker.ofa.iol.unh.edu/bkr/"
export BEAKER_JOB_WHITEBOARD=''
export BEAKER_RECIPE_WHITEBOARD=''
EOF
cat <<"EOF" >/etc/profile.d/beaker.csh
setenv BEAKER "http://beaker.ofa.iol.unh.edu/bkr/"
setenv BEAKER_JOB_WHITEBOARD ''
setenv BEAKER_RECIPE_WHITEBOARD ''
EOF
cat << EOF > /etc/profile.d/rh-env.sh
export LAB_CONTROLLER=beaker.ofa.iol.unh.edu
export DUMPSERVER=""

# Inside Red Hat, the NFSSERVERS variable points to mounts for each of the RHEL
# releases supported, so for example
# rhel5-nfs.blah.foo.bar.redhat.com:/export/home which would then have a
# a directory structure of <fqdn_of_test_machine>/ and in that directory
# would be freeform files that can be used in a test setting.
export NFSSERVERS=""

# The lookaside area is for non-release specific files to be downloaded.
# Generally, they are user space programs, scripts, or tests that are
# expected to work on any install and don't need a per-release area.
# All of the rdma-setup scripts are in this area inside the Red Hat cluster
export LOOKASIDE=http://builder-01.ofa.iol.unh.edu/lookaside

# This is the root directory for downloading build files.  This is not
# like something we'll need inside the OFA cluster as we don't have internal,
# private builds like we do inside Red Hat
export BUILDURL=http://download.eng.bos.redhat.com
EOF
mkdir -p /root/.ssh
cat >>/root/.ssh/authorized_keys <<"__EOF__"
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN6O+038CKOF2sxpsLV68JypHMgdXbmJQkWqdKXnpoZv dledford@linux-ws.n
c.xsintricity.com
__EOF__
restorecon -R /root/.ssh
chmod go-w /root /root/.ssh /root/.ssh/authorized_keys
# Disable rhts-compat for Fedora15/RHEL7 and newer.
cat >> /etc/profile.d/task-overrides-rhts.sh <<END
export RHTS_OPTION_COMPATIBLE=
export RHTS_OPTION_COMPAT_SERVICE=
END
# no snippet data for RedHatEnterpriseLinux7_post
# no snippet data for RedHatEnterpriseLinux_post
%end

[shall1024]

I was doing an automated provision; I'll try a manual one next to see if the results are the same on my end. I will also look into the system_post snippet and see why it is not being added.

I tried to add --nogpgcheck to the end of the epel repo definition and it broke the installation (the installation would never complete). In the anaconda documentation, it seems like --nogpgcheck is not an option, but --nosslverify is an option.

I can try to run another install with --nogpgcheck instead of --nosslverify and can let you know what happens.

[shall1024]

@dledford , the reason that your version did not have the test harness was because I omitted the EPEL repo line from my install. This was because of the issue I mention above; we definitely need some sort of --nogpgcheck option to get the EPEL repo into the provision so the test harness is pulled correctly.

Otherwise, it seems like manual and automatic provisions yield the same results. I'm still looking into why the system_post snippet wouldn't be recognized by the system...I almost wonder if we need to go the route of specifying different system_post files for each distro, but according to Beaker's documentation, we should not have to do that.

[shall1024]

While I work on seeing if I can find any configuration problems that might cause this issue, I created a bug ticket for the Beaker folks, just in case. Nothing on the beaker-project website suggests that for any reason system_post should be ignored, nor do any config files seem to suggest this.

[shall1024]

@dledford , https://github.com/beaker-project/beaker/issues/131#issuecomment-840167372 + some tests I've run today lead me to believe that the global per_osmajor is being ignored because we have one for per_lab, which takes precedence.

[dledford]

The per_lab post snippet can probably be completely deleted, file and all. It's a carbon copy of the pre and I doubt we need it both places. But, if we do, we can always move the per lab post snippet to the global post snippet and merge it with our current global post snippet

On Thu, May 13, 2021 at 12:46 PM shall1024 @.***> wrote:

@dledford https://github.com/dledford , beaker-project/beaker#131 (comment) https://github.com/beaker-project/beaker/issues/131#issuecomment-840167372

• some tests I've run today lead me to believe that the global per_osmajor is being ignored because we have one for per_lab, which takes precedence.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OpenFabrics/fsdp_docs/issues/38#issuecomment-840686845, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIFQFO5JCMCV5XZ5HSQ5GTTNP65RANCNFSM44WKR75Q .

[shall1024]

Sounds great, I will delete the snippet and see if the system_post is pulled. If it is, I plan to close the issue; please let me know if there's another snippet issue I've missed.

[shall1024]

@dledford, the pull request has been created here and all systems should be free for testing, please let us know if you have any issues with any of the test nodes.

I'm going to keep this issue open for now until I confirm that the snippets are correct for all distros. I will test this at a later date when the nodes are free.

[shall1024]

As of 7/1, snippets have been tested for all distros and are confirmed to:

• Add all repos required for test harnesses
• Add all repos required for the packages listed in the snippets we wrote
• Use the correct system_post snippet

All PRs regarding these issues have been accepted, so I am going to close this issue.