binnes
commented
2 years ago I think this is a shell issue. cloud-init commands as strings get passed to the sh shell to run. The cli-tools.yaml runcmd use more advanced shell features, so the if statements are always using the else clause, installing x86_64 binaries on all architectures
Suggested fix is to change the commands to:
runcmd:
- mkdir -p /run/tmp
- export TERRAFORM_VERSION=1.2.4 && curl -Lso /run/tmp/terraform.zip https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_$(if [ `uname -m` = arm64 -o `uname -m` = aarch64 ]; then echo "arm64"; else echo "amd64"; fi).zip && mkdir -p /run/tmp/terraform && cd /run/tmp/terraform && unzip /run/tmp/terraform.zip && sudo mv ./terraform /usr/local/bin && cd - && rm -rf /run/tmp/terraform && rm /run/tmp/terraform.zip
- export TERRAGRUNT_VERSION=0.36.10 && curl -sLo /run/tmp/terragrunt https://github.com/gruntwork-io/terragrunt/releases/download/v${TERRAGRUNT_VERSION}/terragrunt_linux_$(if [ `uname -m` = arm64 -o `uname -m` = aarch64 ]; then echo "arm64"; else echo "amd64"; fi) && chmod +x /run/tmp/terragrunt && sudo mv /run/tmp/terragrunt /usr/local/bin/terragrunt
- export YQ_VERSION=4.25.2 && curl -Lso /run/tmp/yq "https://github.com/mikefarah/yq/releases/download/v${YQ_VERSION}/yq_linux_$(if [ `uname -m` = arm64 -o `uname -m` = aarch64 ]; then echo "arm64"; else echo "amd64"; fi)" && chmod +x /run/tmp/yq && sudo mv /run/tmp/yq /usr/local/bin/yq
- curl -Lo /run/tmp/kubectl "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/$(if [ `uname -m` = arm64 -o `uname -m` = aarch64 ]; then echo "arm64"; else echo "amd64"; fi)/kubectl" && chmod +x /run/tmp/kubectl && sudo mv /run/tmp/kubectl /usr/local/bin
- export OPENSHIFT_CLI_VERSION=4.10 && sudo curl -Lo /usr/local/oc-client.tar.gz https://mirror.openshift.com/pub/openshift-v4/$(if [ `uname -m` = arm64 -o `uname -m` = aarch64 ]; then echo "arm64"; else echo "amd64"; fi)/clients/ocp/stable-${OPENSHIFT_CLI_VERSION}/openshift-client-linux.tar.gz && sudo mkdir /usr/local/oc-client && cd /usr/local/oc-client && tar xzf /usr/local/oc-client.tar.gz && sudo mv ./oc /usr/local/bin && cd - && sudo rm -rf /usr/local/oc-client && sudo rm /usr/local/oc-client.tar.gz
- curl -fsSL https://clis.cloud.ibm.com/install/linux | sh && ibmcloud plugin install container-service -f && ibmcloud plugin install container-registry -f && ibmcloud plugin install observe-service -f && ibmcloud plugin install vpc-infrastructure -f && ibmcloud config --check-version=false
waiting to get confirmation that this resolves the problem as I don't have an ARM based Mac
deleeuwblue
https://github.com/cloud-native-toolkit/sre-utilities/blob/main/cloud-init/cli-tools.yaml
I appreciate the above script was recently updated with improved logic to determine the platform, and download the correct binaries. When I test on Apple M1, I still get amd64 binaries in /usr/local/bin. I did some tests:
I created a new Multipass VM and ran the bootstrap process with the cli-tools.yaml. I ensured my firewall was disabled and cisco VPN not running to ensure network access. I even added the additional runcmds below to help debug. The bootstrap process definitely runs as root. The uname definitely returns aarch64.
However, the binaries (terraform, oc, kubectl, yq etc) in /usr/local/bin are still wrong, and they all report error "-bash: ./oc: cannot execute binary file: Exec format error"
Inside the VM I ran the command below as user ubuntu from /home/ubuntu. It does exactly the same thing as the cli-tools.yaml, only it writes to the home directory instead of /run/tmp (which is owned by root):
The resulting kubectl in /home/ubuntu works fine.
I note that the binaries in /usr/local/bin vs /home/ubuntu have different sizes. If I intentionally download the amd64 version of kubectl, this matches the file size of kubectl in /usr/local/bin. It seems the bootstrap process using cli-tools.yaml still somehow downloads the wrong files.
Thanks