search

operator-framework / operator-sdk

SDK for building Kubernetes applications. Provides high level APIs, useful abstractions, and project scaffolding.
https://sdk.operatorframework.io
Apache License 2.0
7.1k stars 1.73k forks source link

Can operator-sdk run bundle use local image? #6699

Open whg517 opened 4 months ago

whg517 commented 4 months ago

Type of question

Hi, I am writing operator using operator-sdk. Recently I was looking for a solution on how to do e2e testing. I chose to use kind as a temporary test environment.

I started the kind node locally, and then built the operator image and bundle image locally. I used kind load docker-image to load the two images to the kind cluster. When I tried to initialize the test suite environment through operator-sdk run bundle before the test, I found that operator-sdk would pull the images through the network.

In CI environment, these operations are temporary, in order to verify that there is no problem with my logic, I think operator-sdk should first try to use the local bundle image, instead of pulling the remote repository, otherwise you need to provide a test repository to store the temporary image or special version. I think this mechanism is bad for development testing.

I want to ask is this something wrong with me or does operator-sdk run bundle always pull the image of the remote repository?

Question

What did you do?

output:

INFO[0001] trying next host - response was http.StatusNotFound  host=quay.io
FATA[0001] Failed to run bundle: pull bundle image: error pulling image quay.io/zncdata/spark-k8s-operator-bundle:vv0.1.0: error resolving name for image ref quay.io/zncdata/spark-k8s-operator-bundle:vv0.1.0: quay.io/zncdata/spark-k8s-operator-bundle:vv0.1.0: not found

What did you expect to see?

operator-sdk shoule use local image builded with docker, rather then pull bundle.

What did you see instead? Under which circumstances?

above

Environment

Operator type:

golang

Kubernetes cluster type:

$ operator-sdk version

operator-sdk version: "v1.33.0", commit: "542966812906456a8d67cf7284fc6410b104e118", kubernetes version: "v1.27.0", go version: "go1.21.5", GOOS: "darwin", GOARCH: "arm64"

$ go version (if language is Go)

go version go1.21.5 darwin/arm64

$ kubectl version

WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.4", GitCommit:"f89670c3aa4059d6999cb42e23ccb4f0b9a03979", GitTreeState:"clean", BuildDate:"2023-04-12T12:13:53Z", GoVersion:"go1.19.8", Compiler:"gc", Platform:"darwin/arm64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.3", GitCommit:"25b4e43193bcda6c7328a6d147b1fb73a33f1598", GitTreeState:"clean", BuildDate:"2023-06-15T00:38:14Z", GoVersion:"go1.20.5", Compiler:"gc", Platform:"linux/arm64"}

Additional context

whg517 commented 4 months ago

By reading the source code, I found that the mirror implementation logic supports getting the image from the local, but the local mirror logic is not checked at the time of calling.

https://github.com/operator-framework/operator-sdk/blob/4abd483aa4004af83b3c05ddf9d7a7e4d4653cc7/internal/registry/image.go#L30-L72

But at the time of the call, no local judgment is made

https://github.com/operator-framework/operator-sdk/blob/4abd483aa4004af83b3c05ddf9d7a7e4d4653cc7/internal/olm/operator/helpers.go#L37-L40

I was wondering if anyone could fix this?

burmanm commented 3 months ago

I recommend installing local registry like instructed here: https://kind.sigs.k8s.io/docs/user/local-registry/

After that, you can push to that registry and run a local bundle. For example, my quick test:

➜  cass-operator git:(master) ✗ bin/operator-sdk run bundle localhost:5001/k8ssandra/cass-operator-bundle:v1.20.0-dev.faa4d28-20240404 --use-http
INFO[0008] Creating a File-Based Catalog of the bundle "localhost:5001/k8ssandra/cass-operator-bundle:v1.20.0-dev.faa4d28-20240404" 
INFO[0008] Generated a valid File-Based Catalog         
INFO[0016] Created registry pod: 001-k8ssandra-cass-operator-bundle-v1-20-0-dev-faa4d28-20240404 
INFO[0016] Created CatalogSource: cass-operator-catalog 
INFO[0016] OperatorGroup "operator-sdk-og" created      
INFO[0016] Created Subscription: cass-operator-v1-20-0-dev-faa4d28-20240404-sub 
INFO[0023] Approved InstallPlan install-kbnqd for the Subscription: cass-operator-v1-20-0-dev-faa4d28-20240404-sub 
INFO[0023] Waiting for ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" to reach 'Succeeded' phase 
INFO[0024]   Waiting for ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" to appear 
INFO[0026]   Found ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" phase: Pending 
INFO[0028]   Found ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" phase: Installing 
INFO[0089]   Found ClusterServiceVersion "default/cass-operator.v1.20.0-dev.faa4d28-20240404" phase: Succeeded 
INFO[0089] OLM has successfully installed "cass-operator.v1.20.0-dev.faa4d28-20240404" 
➜  cass-operator git:(master) ✗

Personally I always have local registry installed in my kind setups, since it gives more "e2e" feeling in that sense.