search

operator-framework / operator-sdk

SDK for building Kubernetes applications. Provides high level APIs, useful abstractions, and project scaffolding.
https://sdk.operatorframework.io
Apache License 2.0
7.1k stars 1.73k forks source link

quickstart does not work #6681

Open richard-fairthorne opened 4 months ago

richard-fairthorne commented 4 months ago

Bug Report

What did you do?

I followed the quickstart documentation found at https://v1-33-x.sdk.operatorframework.io/docs/building-operators/golang/quickstart/

./trysdk.sh &> trysdk.log

#/bin/bash

set -o xtrace

operator-sdk version

cat /etc/os-release

go version

# EVERYTHING HERE IS COPIED *VERBATIM* FOR THE GO QUICKSTART LOCATED AT
# https://v1-33-x.sdk.operatorframework.io/docs/building-operators/golang/quickstart/

mkdir memcached-operator

cd memcached-operator

operator-sdk init --domain example.com --repo github.com/example/memcached-operator

operator-sdk create api --group cache --version v1alpha1 --kind Memcached --resource --controller

make docker-build docker-push IMG="example.com/memcached-operator:v0.0.1"

What did you expect to see?

It's a quickstart. I expected to see an operator-sdk project start, quickly, preferably, without error.

What did you see instead? Under which circumstances?

I saw segfaults and puppy tears. Here is the output:

trysdk.log


++ operator-sdk version
operator-sdk version: "v1.33.0", commit: "542966812906456a8d67cf7284fc6410b104e118", kubernetes version: "1.27.0", go version: "go1.21.5", GOOS: "linux", GOARCH: "amd64"
++ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
++ go version
go version go1.22.0 linux/amd64
++ mkdir memcached-operator
++ cd memcached-operator
++ operator-sdk init --domain example.com --repo github.com/example/memcached-operator
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.15.0
Update dependencies:
$ go mod tidy
Next: define a resource with:
$ operator-sdk create api
++ operator-sdk create api --group cache --version v1alpha1 --kind Memcached --resource --controller
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
api/v1alpha1/memcached_types.go
api/v1alpha1/groupversion_info.go
internal/controller/suite_test.go
internal/controller/memcached_controller.go
Update dependencies:
$ go mod tidy
Running make:
$ make generate
mkdir -p /home/debian/memcached-operator/bin
test -s /home/debian/memcached-operator/bin/controller-gen && /home/debian/memcached-operator/bin/controller-gen --version | grep -q v0.12.0 || \
GOBIN=/home/debian/memcached-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.12.0
/home/debian/memcached-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa0ba3e]

goroutine 166 [running]: go/types.(Checker).handleBailout(0xc000b7fa00, 0xc000cb9d40) /usr/local/go/src/go/types/check.go:367 +0x88 panic({0xbc6180?, 0x12b0cc0?}) /usr/local/go/src/runtime/panic.go:770 +0x132 go/types.(StdSizes).Sizeof(0x0, {0xdc1618, 0x12b9420}) /usr/local/go/src/go/types/sizes.go:228 +0x31e go/types.(Config).sizeof(...) /usr/local/go/src/go/types/sizes.go:333 go/types.representableConst.func1({0xdc1618?, 0x12b9420?}) /usr/local/go/src/go/types/const.go:76 +0x9e go/types.representableConst({0xdc79b0, 0x1285540}, 0xc000b7fa00, 0x12b9420, 0x0) /usr/local/go/src/go/types/const.go:92 +0x192 go/types.(Checker).arrayLength(0xc000b7fa00, {0xdc5cc8, 0xc0014477e0?}) /usr/local/go/src/go/types/typexpr.go:510 +0x2d3 go/types.(Checker).typInternal(0xc000b7fa00, {0xdc42e8, 0xc001443b90}, 0x0) /usr/local/go/src/go/types/typexpr.go:299 +0x49d go/types.(Checker).definedType(0xc000b7fa00, {0xdc42e8, 0xc001443b90}, 0xc000cb9328?) /usr/local/go/src/go/types/typexpr.go:180 +0x37 go/types.(Checker).varType(0xc000b7fa00, {0xdc42e8, 0xc001443b90}) /usr/local/go/src/go/types/typexpr.go:145 +0x25 go/types.(Checker).structType(0xc000b7fa00, 0xc00145b470, 0xc00145b470?) /usr/local/go/src/go/types/struct.go:113 +0x19f go/types.(Checker).typInternal(0xc000b7fa00, {0xdc4258, 0xc00143d9e0}, 0xc00145eff0) /usr/local/go/src/go/types/typexpr.go:316 +0x1345 go/types.(Checker).definedType(0xc000b7fa00, {0xdc4258, 0xc00143d9e0}, 0xc8fd6b?) /usr/local/go/src/go/types/typexpr.go:180 +0x37 go/types.(Checker).typeDecl(0xc000b7fa00, 0xc00145eff0, 0xc00144c700, 0x0) /usr/local/go/src/go/types/decl.go:615 +0x44d go/types.(Checker).objDecl(0xc000b7fa00, {0xdcceb8, 0xc00145eff0}, 0x0) /usr/local/go/src/go/types/decl.go:197 +0xa7f go/types.(Checker).packageObjects(0xc000b7fa00) /usr/local/go/src/go/types/resolver.go:681 +0x425 go/types.(Checker).checkFiles(0xc000b7fa00, {0xc000fe4a50, 0x3, 0x3}) /usr/local/go/src/go/types/check.go:408 +0x1a5 go/types.(Checker).Files(...) /usr/local/go/src/go/types/check.go:372 sigs.k8s.io/controller-tools/pkg/loader.(loader).typeCheck(0xc000229380, 0xc00052a460) /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/loader.go:286 +0x36a sigs.k8s.io/controller-tools/pkg/loader.(Package).NeedTypesInfo(0xc00052a460) /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/loader.go:99 +0x39 sigs.k8s.io/controller-tools/pkg/loader.(TypeChecker).check(0xc0006726f0, 0xc00052a460) /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/refs.go:268 +0x2b7 sigs.k8s.io/controller-tools/pkg/loader.(TypeChecker).check.func1(0x0?) /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/refs.go:262 +0x53 created by sigs.k8s.io/controller-tools/pkg/loader.(TypeChecker).check in goroutine 20 /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/refs.go:260 +0x1c5 make: *** [Makefile:104: generate] Error 2 Error: failed to create API: unable to run post-scaffold tasks of "base.go.kubebuilder.io/v4": exit status 2 Usage: operator-sdk create api [flags]

Examples:

Create a frigates API with Group: ship, Version: v1beta1 and Kind: Frigate

operator-sdk create api --group ship --version v1beta1 --kind Frigate

Edit the API Scheme

nano api/v1beta1/frigate_types.go

Edit the Controller

nano internal/controller/frigate/frigate_controller.go

Edit the Controller Test

nano internal/controller/frigate/frigate_controller_test.go

Generate the manifests

make manifests

Install CRDs into the Kubernetes cluster using kubectl apply

make install

Regenerate code and run against the Kubernetes cluster configured by ~/.kube/config

make run

Flags: --controller if set, generate the controller without prompting the user (default true) --force attempt to create resource even if it already exists --group string resource Group -h, --help help for api --kind string resource Kind --make make generate if true, run make generate after generating files (default true) --namespaced resource is namespaced (default true) --plural string resource irregular plural form --resource if set, generate the resource without prompting the user (default true) --version string resource Version

Global Flags: --plugins strings plugin keys to be used for this subcommand execution --verbose Enable verbose logging

time="2024-02-15T19:54:48Z" level=fatal msg="failed to create API: unable to run post-scaffold tasks of \"base.go.kubebuilder.io/v4\": exit status 2" ++ make docker-build docker-push IMG=example.com/memcached-operator:v0.0.1 /home/debian/memcached-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa0ba3e]

goroutine 93 [running]: go/types.(Checker).handleBailout(0xc000805800, 0xc00052dd40) /usr/local/go/src/go/types/check.go:367 +0x88 panic({0xbc6180?, 0x12b0cc0?}) /usr/local/go/src/runtime/panic.go:770 +0x132 go/types.(StdSizes).Sizeof(0x0, {0xdc1618, 0x12b9420}) /usr/local/go/src/go/types/sizes.go:228 +0x31e go/types.(Config).sizeof(...) /usr/local/go/src/go/types/sizes.go:333 go/types.representableConst.func1({0xdc1618?, 0x12b9420?}) /usr/local/go/src/go/types/const.go:76 +0x9e go/types.representableConst({0xdc79b0, 0x1285540}, 0xc000805800, 0x12b9420, 0xc00052d4b0) /usr/local/go/src/go/types/const.go:92 +0x192 go/types.(Checker).representation(0xc000805800, 0xc000f9e780, 0x12b9420) /usr/local/go/src/go/types/const.go:256 +0x65 go/types.(Checker).implicitTypeAndValue(0xc000805800, 0xc000f9e780, {0xdc1640, 0xc000132620}) /usr/local/go/src/go/types/expr.go:375 +0x30d go/types.(Checker).assignment(0xc000805800, 0xc000f9e780, {0xdc1640, 0xc000132620}, {0xc93832, 0x14}) /usr/local/go/src/go/types/assignments.go:52 +0x2e5 go/types.(Checker).initConst(0xc000805800, 0xc000bfe900, 0xc000f9e780) /usr/local/go/src/go/types/assignments.go:126 +0x336 go/types.(Checker).constDecl(0xc000805800, 0xc000bfe900, {0xdc4288, 0xc000a19ec0}, {0xdc4288, 0xc000a19ee0}, 0x0) /usr/local/go/src/go/types/decl.go:490 +0x348 go/types.(Checker).objDecl(0xc000805800, {0xdcd098, 0xc000bfe900}, 0x0) /usr/local/go/src/go/types/decl.go:191 +0xa49 go/types.(Checker).packageObjects(0xc000805800) /usr/local/go/src/go/types/resolver.go:693 +0x4dd go/types.(Checker).checkFiles(0xc000805800, {0xc0006aa4b0, 0x5, 0x5}) /usr/local/go/src/go/types/check.go:408 +0x1a5 go/types.(Checker).Files(...) /usr/local/go/src/go/types/check.go:372 sigs.k8s.io/controller-tools/pkg/loader.(loader).typeCheck(0xc000253440, 0xc000892220) /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/loader.go:286 +0x36a sigs.k8s.io/controller-tools/pkg/loader.(Package).NeedTypesInfo(0xc000892220) /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/loader.go:99 +0x39 sigs.k8s.io/controller-tools/pkg/loader.(TypeChecker).check(0xc0007cba40, 0xc000892220) /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/refs.go:268 +0x2b7 sigs.k8s.io/controller-tools/pkg/loader.(TypeChecker).check.func1(0x57?) /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/refs.go:262 +0x53 created by sigs.k8s.io/controller-tools/pkg/loader.(*TypeChecker).check in goroutine 71 /home/debian/go/pkg/mod/sigs.k8s.io/controller-tools@v0.12.0/pkg/loader/refs.go:260 +0x1c5 make: *** [Makefile:100: manifests] Error 2



#### Environment

**Operator type:**

<!-- Uncomment one or more of the following lines corresponding to the language of the operator type -->

/language go
<!-- /language ansible - For ansible operator related bugs, please create the issue in https://github.com/operator-framework/ansible-operator-plugins -->
<!-- /language helm -->

**Kubernetes cluster type:**

<!-- The type of cluster used for testing/deployment, ex. "vanilla", "OpenShift" -->

Didn't get far enough to talk to a cluster, but minikube was running. 

`$ operator-sdk version`

<!-- Insert the output of `operator-sdk version` here. -->

operator-sdk version: "v1.33.0", commit: "542966812906456a8d67cf7284fc6410b104e118", kubernetes version: "1.27.0", go version: "go1.21.5", GOOS: "linux", GOARCH: "amd64"

`$ go version` (if language is Go)

<!-- Insert the output of `go version` here -->

go version go1.22.0 linux/amd64

`$ kubectl version`

<!-- Insert the output of `kubectl version` here -->

Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3

#### Possible Solution

<!-- Only if you have suggestions on a fix for the bug -->

#### Additional context

<!-- Add any other context about the problem here. -->

These logs are from amd64 hardware. The exact same thing happens on apple silicon, docker desktop on apple silicon, and docker desktop amd64-rosetta on Apple silicon.
krutsko commented 4 months ago

downgrade Go to 1.21 works for me

richard-fairthorne commented 4 months ago

Downgrade to Go 1.21 Works for me as well, on all platforms.

Go 1.22 is stable. This could be:

Potential fixes could include:

Does anyone have the skills and availability to spearhead this?

olivercodes commented 4 months ago

After fighting with this for a couple of days, I came across this thread. I can confirm the same issue with go 1.22 on both a Mac M1 (ARM64) and on Linux amd64.

Downgrading to 1.21 on both systems resolved the problem.

s3rj1k commented 4 months ago

Would be nice to have official operator-sdk Docker image updated, so that it would have Go 1.21 compiler in it and binary compiled with the same version. Had to manually rebuild image with version change in here https://github.com/operator-framework/operator-sdk/blob/master/images/operator-sdk/Dockerfile#L22.

Having a working Docker image can at least be considered a quick workaround without needing to downgrade the local Golang runtime version.

richard-fairthorne commented 4 months ago

Would be nice to have official operator-sdk Docker image updated, so that it would have Go 1.21 compiler in it and binary compiled with the same version.

This could probably be useful. If you think you have strong use-case support, you should consider opening a ticket for it.

shanduur commented 4 months ago

Looks like might be related to https://github.com/kubernetes-sigs/controller-tools/issues/851 ?

Edit: On go1.22 and controller-tools@v0.12.1 the issue is present (running go test ./...), while running tests on latest release does not fail.

richard-fairthorne commented 4 months ago

Edit: On go1.22 and controller-tools@v0.12.1 the issue is present (running go test ./...), while running tests on latest release does not fail.

For clarity, does that rule out https://github.com/kubernetes-sigs/controller-tools/issues/851 as the culprit?

shanduur commented 4 months ago

@richard-fairthorne other way around. The panic is caused by the version of controller-tools currently used in operator-sdk. Updating it probably should fix the issue.

Hayden-Chang commented 4 months ago

Upgrading go from 1.21.1 to 1.21.7 works for me

harshvijaythakkar commented 4 months ago

For MAC darwin/arm64 users following will work:

  1. Your go version should be 1.27

    go version go1.21.7 darwin/arm64

  2. No need to pass --plugins=go/v4-alpha argument because kubebuilder documentation has made go/v4 as default plugin and it supports MAC darwin/arm64

    operator-sdk init --domain example.com --repo <your_repo_url>

  3. Create API 

    operator-sdk create api --group cache --version v1alpha1 --kind Memcached --resource --controller

  4. Generate CRD

    make generate
mkfdoherty commented 3 months ago

This error appears connected to the controller-gen binary. A similar panic was documented in an issue on the controller-tools repo:

This happened with controller-gen v0.13.0. When I upgraded to v0.14.0, it disappeared.

https://github.com/kubernetes-sigs/controller-tools/issues/888

The fix involves updating the CONTROLLER_TOOLS_VERSION in your Makefile from v0.13.0 to v0.14.0. This will rebuild a working controller-gen binary even with the latest Go version (v1.22.1). Ensure you rm ./bin/controller-gen so that the make will build a new binary when you make generate.

jberkhahn commented 3 months ago

We discussed this and we need to bump several dependencies in several places in our end to fix this. We'll be targeting this for our next release. Until then, our recommendation is to stick to go 1.21.

acornett21 commented 3 months ago
alejandrojnm commented 2 months ago

Yes with controller-gen v0.14.0 work the last version of operator-sdk with v0.13.0 fail

controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x1033ec604]

goroutine 117 [running]:
go/types.(*Checker).handleBailout(0x1400062e200, 0x14000861d18)
        /usr/local/go/src/go/types/check.go:367 +0x9c
panic({0x1036c5a60?, 0x103c5caf0?})
        /usr/local/go/src/runtime/panic.go:770 +0x124
go/types.(*StdSizes).Sizeof(0x0, {0x103790990, 0x103c65300})
        /usr/local/go/src/go/types/sizes.go:228 +0x314
go/types.(*Config).sizeof(...)
        /usr/local/go/src/go/types/sizes.go:333
go/types.representableConst.func1({0x103790990?, 0x103c65300?})
        /usr/local/go/src/go/types/const.go:76 +0x9c
go/types.representableConst({0x103796c30, 0x103c30240}, 0x1400062e200, 0x103c65300, 0x0)
        /usr/local/go/src/go/types/const.go:92 +0x138
go/types.(*Checker).arrayLength(0x1400062e200, {0x103794f78, 0x14000e243a0?})
        /usr/local/go/src/go/types/typexpr.go:510 +0x238
go/types.(*Checker).typInternal(0x1400062e200, {0x103793598, 0x14000c45980}, 0x0)
        /usr/local/go/src/go/types/typexpr.go:299 +0x3bc
go/types.(*Checker).definedType(0x1400062e200, {0x103793598, 0x14000c45980}, 0x102e64bac?)
        /usr/local/go/src/go/types/typexpr.go:180 +0x2c
go/types.(*Checker).varType(0x1400062e200, {0x103793598, 0x14000c45980})
        /usr/local/go/src/go/types/typexpr.go:145 +0x2c
go/types.(*Checker).structType(0x1400062e200, 0x140012ca090, 0x140012ca090?)
        /usr/local/go/src/go/types/struct.go:113 +0x128
go/types.(*Checker).typInternal(0x1400062e200, {0x103793508, 0x140010e5860}, 0x14001232410)
        /usr/local/go/src/go/types/typexpr.go:316 +0xed0
go/types.(*Checker).definedType(0x1400062e200, {0x103793508, 0x140010e5860}, 0x103500528?)
        /usr/local/go/src/go/types/typexpr.go:180 +0x2c
go/types.(*Checker).typeDecl(0x1400062e200, 0x14001232410, 0x14000c42940, 0x0)
        /usr/local/go/src/go/types/decl.go:615 +0x39c
go/types.(*Checker).objDecl(0x1400062e200, {0x10379bb40, 0x14001232410}, 0x0)
        /usr/local/go/src/go/types/decl.go:197 +0x880
go/types.(*Checker).packageObjects(0x1400062e200)
        /usr/local/go/src/go/types/resolver.go:681 +0x3c0
go/types.(*Checker).checkFiles(0x1400062e200, {0x14000ea8738, 0x3, 0x3})
        /usr/local/go/src/go/types/check.go:408 +0x164
go/types.(*Checker).Files(...)
        /usr/local/go/src/go/types/check.go:372
sigs.k8s.io/controller-tools/pkg/loader.(*loader).typeCheck(0x140002713b0, 0x140001f6e80)
        /Users/alejandrojnm/Project/go/pkg/mod/sigs.k8s.io/controller-tools@v0.13.0/pkg/loader/loader.go:286 +0x2d8
sigs.k8s.io/controller-tools/pkg/loader.(*Package).NeedTypesInfo(0x140001f6e80)
        /Users/alejandrojnm/Project/go/pkg/mod/sigs.k8s.io/controller-tools@v0.13.0/pkg/loader/loader.go:99 +0x44
sigs.k8s.io/controller-tools/pkg/loader.(*TypeChecker).check(0x140007c0570, 0x140001f6e80)
        /Users/alejandrojnm/Project/go/pkg/mod/sigs.k8s.io/controller-tools@v0.13.0/pkg/loader/refs.go:268 +0x304
sigs.k8s.io/controller-tools/pkg/loader.(*TypeChecker).check.func1(0x22?)
        /Users/alejandrojnm/Project/go/pkg/mod/sigs.k8s.io/controller-tools@v0.13.0/pkg/loader/refs.go:262 +0x58
created by sigs.k8s.io/controller-tools/pkg/loader.(*TypeChecker).check in goroutine 103
        /Users/alejandrojnm/Project/go/pkg/mod/sigs.k8s.io/controller-tools@v0.13.0/pkg/loader/refs.go:260 +0x230
make: *** [generate] Error 2
Error: failed to create API: unable to run post-scaffold tasks of "base.go.kubebuilder.io/v4": exit status 2
franciscoprin commented 2 months ago

I'm having this problem with go1.21.9

$ go version               
go version go1.21.9 darwin/arm64
harshitsinghvi22 commented 1 month ago

i have the below go version and it works fine when I UPDATED THE CONTROLLER-GEN VERSION in the Makefile to >=0.14.0 go version go1.22.3 darwin/amd64

qfaure commented 1 month ago

i have the below go version and it works fine when I UPDATED THE CONTROLLER-GEN VERSION in the Makefile to >=0.14.0 go version go1.22.3 darwin/amd64

Work for me. Thanks @harshitsinghvi22
CONTROLLER_TOOLS_VERSION ?= v0.14.0 Don't forget to delete binary in /bin if you previously launch make with versin 0.13.0

cabrinha commented 1 month ago 
CONTROLLER_TOOLS_VERSION ?= v0.14.0 # also tried v0.15.0

go version go1.22.3 darwin/arm64

operator-sdk version: "v1.34.2", commit: "81dd3cb24b8744de03d312c1ba23bfc617044005", kubernetes version: "v1.28.0", go version: "go1.21.10", GOOS: "darwin", GOARCH: "arm64"

make generate works fine, but make manifests:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x20 pc=0x100c2da64]