[BUG] Error on M1 arm64 raising up a CRC instance

[jparrill]

General information

• OS: macOS M1 arm
• Did you run crc setup before starting it: Yes (actually it fails on the setup
• Running CRC on: Laptop

CRC version

CRC version: 2.20.0+f3a947
OpenShift version: 4.13.0
Podman version: 4.4.4

CRC status

DEBU CRC version: 2.20.0+f3a947
DEBU OpenShift version: 4.13.0
DEBU Podman version: 4.4.4
DEBU Running 'crc status'
crc does not seem to be setup correctly, have you run 'crc setup'?

CRC config

- consent-telemetry                     : no
- preset                                : microshift

Host Operating System

ProductName:        macOS
ProductVersion:     13.4
BuildVersion:       22F66

Steps to reproduce

1. Follow the troubleshoot guide (in the previous version didn't raise up the microshift cluster)
2. crc stop: ✅
3. crc delete: ✅
4. crc cleanup: ✅
5. crc setup: ❌

• I've also tried deleting the podman-machine

  1. podman machine delete: ✅
  2. podman machine rm: ✅
  3. podman machine init: ✅
  4. podman machine info: ✅

Host:
  Arch: arm64
  CurrentMachine: podman-machine-default
  DefaultMachine: podman-machine-default
  EventsDir: /var/folders/jl/ys9r5fnn1xd5d5gfrqkf7s1h0000gn/T/podman-run--1/podman
  MachineConfigDir: /Users/jparrill/.config/containers/podman/machine/qemu
  MachineImageDir: /Users/jparrill/.local/share/containers/podman/machine/qemu
  MachineState: Running
  NumberOfMachines: 1
  OS: darwin
  VMType: qemu
Version:
  APIVersion: 4.5.1
  Built: 1685113812
  BuiltTime: Fri May 26 17:10:12 2023
  GitCommit: 9eef30051c83f62816a1772a743e5f1271b196d7
  GoVersion: go1.20.4
  Os: darwin
  OsArch: darwin/arm64
  Version: 4.5.1

5. crc stop: ✅
6. crc delete: ✅
7. crc cleanup: ✅
8. crc setup: ❌

INFO Using bundle path /Users/jparrill/.crc/cache/crc_microshift_vfkit_4.13.0_amd64.crcbundle
INFO Checking if running as non-root
INFO Checking if crc-admin-helper executable is cached
INFO Checking if running on a supported CPU architecture
INFO Checking if crc executable symlink exists
INFO Creating symlink for crc executable
INFO Checking minimum RAM requirements
INFO Checking if running emulated on a M1 CPU
This version of CRC for AMD64/Intel64 CPUs is unsupported on Apple M1 hardware

Expected

• Raise up the Microshift instance properly

Actual

Error with This version of CRC for AMD64/Intel64 CPUs is unsupported on Apple M1 hardware

Logs

Σ k8s-dev-env git:(main) crc setup
INFO Using bundle path /Users/jparrill/.crc/cache/crc_microshift_vfkit_4.13.0_amd64.crcbundle
INFO Checking if running as non-root
INFO Checking if crc-admin-helper executable is cached
INFO Checking if running on a supported CPU architecture
INFO Checking if crc executable symlink exists
INFO Creating symlink for crc executable
INFO Checking minimum RAM requirements
INFO Checking if running emulated on a M1 CPU
This version of CRC for AMD64/Intel64 CPUs is unsupported on Apple M1 hardware

[cfergeau]

This would happen if you try to run an amd64 crc binary on the M1. I'm under the impression you built crc from source? Make sure you run either the universal binary, or the arm64 one.

[jparrill]

Nope, I've downloaded from here: https://developers.redhat.com/content-gateway/rest/mirror/pub/openshift-v4/clients/crc/2.20.0

The binary is ARM64 compatible:

λ k8s-dev-env git:(main) which crc
/usr/local/bin/crc
λ k8s-dev-env git:(main) file /usr/local/bin/crc
/usr/local/bin/crc: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
/usr/local/bin/crc (for architecture x86_64):   Mach-O 64-bit executable x86_64
/usr/local/bin/crc (for architecture arm64):    Mach-O 64-bit executable arm64

[cfergeau]

Hmm this is getting mysterious, I just downloaded the dmg on my M1,

% crc version
CRC version: 2.20.0+f3a947
OpenShift version: 4.13.0
Podman version: 4.4.4

% which crc
/usr/local/bin/crc

% file /usr/local/bin/crc
/usr/local/bin/crc: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
/usr/local/bin/crc (for architecture x86_64):   Mach-O 64-bit executable x86_64
/usr/local/bin/crc (for architecture arm64):    Mach-O 64-bit executable arm64

% shasum -a 256 /usr/local/bin/crc 
53395fb1a2a5b733f5d14734efb3b908462c27e1d91da5e6bde4567bb9eb1f02  /usr/local/bin/crc

 % crc setup
INFO Using bundle path /Users/teuf/.crc/cache/crc_microshift_vfkit_4.13.0_arm64.crcbundle 
INFO Checking if running as non-root              
INFO Checking if crc-admin-helper executable is cached 
INFO Checking if running on a supported CPU architecture 
INFO Checking if crc executable symlink exists    
INFO Checking minimum RAM requirements            
INFO Checking if running emulated on a M1 CPU     
INFO Checking if vfkit is installed               
INFO Checking if CRC bundle is extracted in '$HOME/.crc' 
INFO Checking if /Users/teuf/.crc/cache/crc_microshift_vfkit_4.13.0_arm64.crcbundle exists 
INFO Checking if old launchd config for tray and/or daemon exists 
INFO Checking if crc daemon plist file is present and loaded 
INFO Adding crc daemon plist file and loading it  
Your system is correctly setup for using CRC. Use 'crc start' to start the instance

% crc start
INFO Checking if running as non-root              
INFO Checking if crc-admin-helper executable is cached 
INFO Checking if running on a supported CPU architecture 
INFO Checking if crc executable symlink exists    
INFO Checking minimum RAM requirements            
INFO Checking if running emulated on a M1 CPU     
INFO Checking if vfkit is installed               
INFO Checking if old launchd config for tray and/or daemon exists 
INFO Checking if crc daemon plist file is present and loaded 
INFO Loading bundle: crc_microshift_vfkit_4.13.0_arm64... 
CRC requires a pull secret to download content from Red Hat.
You can copy it from the Pull Secret section of https://console.redhat.com/openshift/create/local.

[cfergeau]

Just noticed in your output:

INFO Using bundle path /Users/jparrill/.crc/cache/crc_microshift_vfkit_4.13.0_amd64.crcbundle

so it's also using an amd64 bundle instead of an arm64 one?

[jparrill]

Yep, do you know how should I set the bundle in the setup with the --bundle flag? I mean the right crc bundle URL

UPDATE:

Σ k8s-dev-env git:(main) crc setup -b /Users/jparrill/.crc/cache/crc_microshift_vfkit_4.13.0_arm64.crcbundle
WARN Using crc_microshift_vfkit_4.13.0_arm64.crcbundle bundle, but crc_microshift_vfkit_4.13.0_amd64.crcbundle is expected for this release
INFO Using bundle path /Users/jparrill/.crc/cache/crc_microshift_vfkit_4.13.0_arm64.crcbundle
INFO Checking if running as non-root
INFO Checking if crc-admin-helper executable is cached
INFO Checking if running on a supported CPU architecture
INFO Checking if crc executable symlink exists
INFO Checking minimum RAM requirements
INFO Checking if running emulated on a M1 CPU
This version of CRC for AMD64/Intel64 CPUs is unsupported on Apple M1 hardware

[praveenkumar]

@cfergeau which version of MacOS you have? @jparrill mentioned 13.4 I have older 13.3 and not able to reproduce, will upgrade and check.

[cfergeau]

Yep, do you know how should I set the bundle in the setup with the --bundle flag? I mean the right crc bundle URL

Let's first try without specifying a bundle. crc delete; crc cleanup; crc config set preset microshift; crc setup; crc start

[cfergeau]

But the binary you are running really looks like an amd64 binary since it expects an _amd64 bundle, not an arm64 one.

[jparrill]

I've downloaded from here https://developers.redhat.com/content-gateway/rest/mirror/pub/openshift-v4/clients/crc/2.20.0 as I mentioned, the file command executed before shows that this binary could be executed in amd64 and arm64. It's there any other build for arm64 I'm not aware of?

If not I will build one by myself, just to test it.

[cfergeau]

I downloaded/installed exactly the same .pkg file, which looks exactly like yours. Can you double-check the file sha256sum to be sure we are using the same one? And test using the full path to the binary, just in case?

[jparrill]

λ k8s-dev-env git:(main) sha256sum $(which crc)
53395fb1a2a5b733f5d14734efb3b908462c27e1d91da5e6bde4567bb9eb1f02  /usr/local/bin/crc

[praveenkumar]

I tested in the 13.4 and works without any issue and download correct bundle (arm64 one).

[jparrill]

Do you have the URL handy? Podman-Desktop download the client I’ve mentioned

[cfergeau]

Could your environment be forcing somehow the use of x86_64 binaries instead of m1 binaries? I've downloaded the client you've mentioned on my M1 and it worked just fine.

[cfergeau]

Reading https://stackoverflow.com/questions/75229976/what-does-arch-x86-64-zsh-on-m1-mac-do-internally it's actually possible to force the use of an amd64 bnary instead of the native one through the use of the arch command. I'm now wondering if the shell you are using could have been started as an x86_64 one instead of being native?

[praveenkumar]

We got more info from @jparrill (thanks for your time) and found out he is using iterm2 which is running in rosetta mode so the arch on the terminal shows i386 and even uname -a shows

Darwin acheron.local 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun  8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000 x86_64

We need to figure out how to catch something like this as part of preflight check and provide a warning to use arch -arch arm64 crc setup

[cfergeau]

We need to figure out how to catch something like this as part of preflight check

We already catch running the x86_64 binary on an M1, for what you describe we would need a way to know that the crc binary is a universal one. Or we keep the current check and only change the error message, and point to a webpage with more explanations.