kubernetes / minikube

Run Kubernetes locally
https://minikube.sigs.k8s.io/
Apache License 2.0
28.77k stars 4.81k forks source link

nerdctl supports more arches than just amd64 and arm64 #18528

Open evanskinner opened 3 months ago

evanskinner commented 3 months ago

What Happened?

At https://github.com/kubernetes/minikube/blob/89e46869f5308c60af938fb45ee04d9ab33779ed/cmd/minikube/cmd/docker-env.go#L312 and https://github.com/kubernetes/minikube/blob/89e46869f5308c60af938fb45ee04d9ab33779ed/cmd/minikube/cmd/docker-env.go#L678 the docker-env command refuses to run unless the arch is amd64 or arm64 but in fact there are release for all minikubve supported arches: https://github.com/containerd/nerdctl/releases/tag/v1.7.5

Attach the log file

01:36:22 * minikube profile was successfully set to apic 01:36:23 X Exiting due to MK_USAGE: the docker-env command only supports amd64 & arm64 architectures 01:36:23 Failed to start minikube correctly, not proceeding.

Operating System

Redhat/Fedora

Driver

Docker

afbjorklund commented 3 months ago

minikube normally does not want to run in emulation, and amd64 and arm64 are still the most common host arch

But I don't think there needs to be an artificial limitation added, just use what is available... Which one do you use?

afbjorklund commented 3 months ago

Note that it says nerdctld, as opposed to nerdctl. But it's mostly a question of adding some more build targets...

https://github.com/afbjorklund/nerdctld/blob/main/Makefile#L33

evanskinner commented 3 months ago

@afbjorklund Thanks for the quick reply. Ah yes I was looking at the wrong repo, sorry missed the d on the end, but that is good to know that nerdctld could also be built for other arches. I am running on s390x hardware and will also be running on ppc64le hardware so I'm not running any emulation. If those build targets could be added and the check allow them to be used that would be pretty awesome.

afbjorklund commented 3 months ago

Should be a question of adding some more GOARCH targets, and using the go cross-compilation.

Unlike OBS, nfpm does not need to start a new virtual machine in order to package a binary... But there is no support for s390x or ppc64le in Lima, so I don't think there is any way to test it?

Besides the mentioned mainframe targets, I think there is also arm32 (arm/7) and RISC-V (riscv64).

afbjorklund commented 3 months ago

The arm32 target is a bit odd, it needs to be GOARCH=arm GOARM=7 for go but arm7 for nfpm.

The other targets seem OK, but I seem to have misplaced my old lima branch with the arch support.

afbjorklund commented 3 months ago

Found the old discussion at

And still no testing on Fedora

afbjorklund commented 3 months ago

Added 4 new architectures, feel free to verify on your end:

https://github.com/afbjorklund/nerdctld/releases/tag/v0.6.0

evanskinner commented 3 months ago

Thanks very much @afbjorklund. I will try to get to that ASAP, but it might take me a few weeks to get back to it.

evanskinner commented 4 days ago

Sorry for the huge delay. Tested the s390x and ppc64le version and it seems to start and listen fine, with the same output in both architectures:

# ./nerdctld --addr unix://0.0.0.0:4567 --debug
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] HEAD   /_ping                    --> main.setupRouter.func1 (3 handlers)
[GIN-debug] GET    /_ping                    --> main.setupRouter.func2 (3 handlers)
[GIN-debug] GET    /:ver/version             --> main.setupRouter.func3 (3 handlers)
[GIN-debug] GET    /:ver/info                --> main.setupRouter.func4 (3 handlers)
[GIN-debug] GET    /:ver/images/json         --> main.setupRouter.func5 (3 handlers)
[GIN-debug] GET    /:ver/images/:name/json   --> main.setupRouter.func6 (3 handlers)
[GIN-debug] GET    /:ver/images/:name/history --> main.setupRouter.func7 (3 handlers)
[GIN-debug] POST   /:ver/images/:name/tag    --> main.setupRouter.func8 (3 handlers)
[GIN-debug] POST   /:ver/images/:name/push   --> main.setupRouter.func9 (3 handlers)
[GIN-debug] POST   /:ver/images/create       --> main.setupRouter.func10 (3 handlers)
[GIN-debug] DELETE /:ver/images/*name        --> main.setupRouter.func11 (3 handlers)
[GIN-debug] POST   /:ver/images/load         --> main.setupRouter.func12 (3 handlers)
[GIN-debug] GET    /:ver/images/get          --> main.setupRouter.func13 (3 handlers)
[GIN-debug] GET    /:ver/containers/json     --> main.setupRouter.func14 (3 handlers)
[GIN-debug] GET    /:ver/containers/:name/json --> main.setupRouter.func15 (3 handlers)
[GIN-debug] GET    /:ver/containers/:name/logs --> main.setupRouter.func16 (3 handlers)
[GIN-debug] GET    /:ver/volumes             --> main.setupRouter.func17 (3 handlers)
[GIN-debug] GET    /:ver/volumes/:name       --> main.setupRouter.func18 (3 handlers)
[GIN-debug] GET    /:ver/networks            --> main.setupRouter.func19 (3 handlers)
[GIN-debug] GET    /:ver/networks/:name      --> main.setupRouter.func20 (3 handlers)
[GIN-debug] GET    /:ver/system/df           --> main.setupRouter.func21 (3 handlers)
[GIN-debug] POST   /:ver/build               --> main.setupRouter.func22 (3 handlers)
[GIN-debug] POST   /:ver/build/prune         --> main.setupRouter.func23 (3 handlers)
[GIN-debug] Listening and serving HTTP on unix:/0.0.0.0:4567