search

equinix / terraform-equinix-metal-anthos-on-baremetal

Terraform module for quick deployment of baremetal Anthos on Equinix Metal
https://registry.terraform.io/modules/equinix/anthos-on-baremetal
Apache License 2.0
26 stars 24 forks source link

Make this work with all 4x supported OSes #4

Open c0dyhi11 opened 3 years ago

c0dyhi11 commented 3 years ago

This should work with:

This has only been tested on Ubuntu 20.04

c0dyhi11 commented 3 years ago

The only OS left is rhel_8 It need some love. Here is the output of trying to run the same code that works on CentOS_8:

null_resource.prep_anthos_cluster (remote-exec): Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
null_resource.prep_anthos_cluster (remote-exec): Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
null_resource.prep_anthos_cluster (remote-exec): setenforce: SELinux is disabled
null_resource.prep_anthos_cluster (remote-exec):   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
null_resource.prep_anthos_cluster (remote-exec):                                  Dload  Upload   Total   Spent    Left  Speed
null_resource.prep_anthos_cluster (remote-exec):   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
null_resource.prep_anthos_cluster (remote-exec): 100  1919  100  1919    0     0   8566      0 --:--:-- --:--:-- --:--:--  8566
null_resource.prep_anthos_cluster (remote-exec): [google-cloud-sdk]
null_resource.prep_anthos_cluster (remote-exec): name=Google Cloud SDK
null_resource.prep_anthos_cluster (remote-exec): baseurl=https://packages.cloud.google.com/yum/repos/cloud-sdk-el7-x86_64
null_resource.prep_anthos_cluster (remote-exec): enabled=1
null_resource.prep_anthos_cluster (remote-exec): gpgcheck=1
null_resource.prep_anthos_cluster (remote-exec): repo_gpgcheck=1
null_resource.prep_anthos_cluster (remote-exec): gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
null_resource.prep_anthos_cluster (remote-exec):     https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
null_resource.prep_anthos_cluster (remote-exec): Updating Subscription Management repositories.
null_resource.prep_anthos_cluster (remote-exec): Unable to read consumer identity
null_resource.prep_anthos_cluster (remote-exec): This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
null_resource.prep_anthos_cluster (remote-exec): Docker  ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Docker  111 kB/s | 3.8 kB     00:00
null_resource.prep_anthos_cluster (remote-exec): Google  ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Google  1.5 kB/s | 577  B     00:00 ETA
null_resource.prep_anthos_cluster (remote-exec): Google  692  B/s | 454  B     00:00
null_resource.prep_anthos_cluster (remote-exec): Google  ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Google   17 kB/s | 1.8 kB     00:00
null_resource.prep_anthos_cluster (remote-exec): Importing GPG key 0xA7317B0F:
null_resource.prep_anthos_cluster (remote-exec):  Userid     : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
null_resource.prep_anthos_cluster (remote-exec):  Fingerprint: D0BC 747F D8CA F711 7500 D6FA 3746 C208 A731 7B0F
null_resource.prep_anthos_cluster (remote-exec):  From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
null_resource.prep_anthos_cluster (remote-exec): Importing GPG key 0xBA07F4FB:
null_resource.prep_anthos_cluster (remote-exec):  Userid     : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
null_resource.prep_anthos_cluster (remote-exec):  Fingerprint: 54A6 47F9 048D 5688 D7DA 2ABE 6A03 0B21 BA07 F4FB
null_resource.prep_anthos_cluster (remote-exec):  From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
null_resource.prep_anthos_cluster (remote-exec): Google  ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Google  6.7 kB/s | 975  B     00:00
null_resource.prep_anthos_cluster (remote-exec): Importing GPG key 0x3E1BA8D5:
null_resource.prep_anthos_cluster (remote-exec):  Userid     : "Google Cloud Packages RPM Signing Key <gc-team@google.com>"
null_resource.prep_anthos_cluster (remote-exec):  Fingerprint: 3749 E1BA 95A8 6CE0 5454 6ED2 F09C 394C 3E1B A8D5
null_resource.prep_anthos_cluster (remote-exec):  From       : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
null_resource.prep_anthos_cluster (remote-exec): Google  ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Google  ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Google  ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Google   45 kB/s |  14 kB     00:02 ETA
null_resource.prep_anthos_cluster (remote-exec): Google  1.2 MB/s | 5.8 MB     00:13 ETA
null_resource.prep_anthos_cluster (remote-exec): Google   14 MB/s |  22 MB     00:01
null_resource.prep_anthos_cluster: Still creating... [10s elapsed]
null_resource.prep_anthos_cluster (remote-exec): Red Hat ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Red Hat ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Red Hat 8.2 MB/s | 9.2 MB     00:01
null_resource.prep_anthos_cluster (remote-exec): Red Hat ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Red Hat 124 MB/s |  21 MB     00:00
null_resource.prep_anthos_cluster (remote-exec): Red Hat ---  B/s |   0  B     --:-- ETA
null_resource.prep_anthos_cluster (remote-exec): Red Hat  18 MB/s | 843 kB     00:00
null_resource.prep_anthos_cluster (remote-exec): Package iptables-1.8.2-9.el8.x86_64 is already installed.
null_resource.prep_anthos_cluster (remote-exec): No match for argument: python3
null_resource.prep_anthos_cluster (remote-exec): Error: Unable to find a match
null_resource.prep_anthos_cluster (remote-exec): Failed to enable unit: Unit file docker.service does not exist.
null_resource.prep_anthos_cluster (remote-exec):   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
null_resource.prep_anthos_cluster (remote-exec):                                  Dload  Upload   Total   Spent    Left  Speed
null_resource.prep_anthos_cluster (remote-exec):   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
null_resource.prep_anthos_cluster (remote-exec): 100 41.9M  100 41.9M    0     0  87.3M      0 --:--:-- --:--:-- --:--:-- 87.3M
null_resource.prep_anthos_cluster (remote-exec): /root/baremetal/pre_reqs.sh: line 66: gcloud: command not found
null_resource.prep_anthos_cluster (remote-exec): /root/baremetal/pre_reqs.sh: line 67: gsutil: command not found
null_resource.prep_anthos_cluster (remote-exec): chmod: cannot access 'bmctl': No such file or directory
null_resource.prep_anthos_cluster (remote-exec): /root/baremetal/pre_reqs.sh: line 70: ./bmctl: No such file or directory
null_resource.prep_anthos_cluster (remote-exec): sed: can't read /root/baremetal/bmctl-workspace/cody-1awt4/cody-1awt4.yaml: No such file or directory
null_resource.prep_anthos_cluster (remote-exec): sed: can't read /root/baremetal/bmctl-workspace/cody-1awt4/cody-1awt4.yaml: No such file or directory
null_resource.prep_anthos_cluster (remote-exec): sed: can't read /root/baremetal/bmctl-workspace/cody-1awt4/cody-1awt4.yaml: No such file or directory
null_resource.prep_anthos_cluster (remote-exec): sed: can't read /root/baremetal/bmctl-workspace/cody-1awt4/cody-1awt4.yaml: No such file or directory
null_resource.prep_anthos_cluster (remote-exec): sed: can't read /root/baremetal/bmctl-workspace/cody-1awt4/cody-1awt4.yaml: No such file or directory
joshpadilla commented 3 years ago

Consider: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/selinux_users_and_administrators_guide/sect-security-enhanced_linux-enabling_and_disabling_selinux-dracut-parameters

On boot, you can set several kernel parameters to change the way SELinux runs: enforcing=0 Setting this parameter causes the system to start in permissive mode, which is useful when troubleshooting issues. Using permissive mode might be the only option to detect a problem if your file system is too corrupted. Moreover, in permissive mode, the system continues to create the labels correctly.

Could we add a kernel parameter at boot time somewhere here: https://github.com/c0dyhi11/baremetal-anthos/blob/2677bf9ee009f124b06ca43496dae541b2d762c3/templates/user_data.sh#L7

joshpadilla commented 3 years ago

1) Launched RHEL 8.0 c3.medium.x86 in SJC1 on EQMetal 2) By default in EQMetal /etc/selinux/config file exists (disabled)

[root@rhel8-selinux-test selinux]# cat /etc/selinux/config 
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted
c0dyhi11 commented 3 years ago

@joshpadilla the issue with SelInux seemed to be on CentOS. The RHEL is that I couldn’t install certain packages. 


This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.

...

No match for argument: python3
joshpadilla commented 3 years ago

Registered my test host:

sudo subscription-manager remove --all
sudo subscription-manager unregister
sudo subscription-manager clean

sudo subscription-manager register
sudo subscription-manager refresh
sudo subscription-manager attach --auto

[root@jcp-rhel8-selinux-test rhsm]# cat rhsm.conf
# Red Hat Subscription Manager Configuration File:

# Unified Entitlement Platform Configuration
[server]
# Server hostname:
hostname = subscription.rhsm.redhat.com

# Server prefix:
prefix = /subscription

# Server port:
port = 443

# Set to 1 to disable certificate validation:
insecure = 0

# Set the depth of certs which should be checked
# when validating a certificate
ssl_verify_depth = 3

# an http proxy server to use
proxy_hostname =

# The scheme to use for the proxy when updating repo definitions, if needed
# e.g. http or https
proxy_scheme = http
c0dyhi11 commented 3 years ago

@joshpadilla do you need a license?

joshpadilla commented 3 years ago

Yes, can use a free dev license or one for the Eng/CPE teams, we have 20 currently for CI needs and can increase. I think there's a better way to do the host licensing in a non-interactive way. Still reviewing that.