search

kubernetes / sample-controller

Repository for sample controller. Complements sample-apiserver
Apache License 2.0
3.14k stars 1.08k forks source link

The update-codegen script does not automatically generate files & directories #73

Closed max-rocket-internet closed 1 year ago

max-rocket-internet commented 4 years ago

To quote the README:

The update-codegen script will automatically generate the following files & directories:

  • pkg/apis/samplecontroller/v1alpha1/zz_generated.deepcopy.go
  • pkg/generated/

But:

$ git clone https://github.com/kubernetes/sample-controller.git
$ cd sample-controller
$ rm -f pkg/apis/samplecontroller/v1alpha1/zz_generated.deepcopy.go
$ ./hack/update-codegen.sh
Generating deepcopy funcs
Generating clientset for samplecontroller:v1alpha1 at k8s.io/sample-controller/pkg/generated/clientset
Generating listers for samplecontroller:v1alpha1 at k8s.io/sample-controller/pkg/generated/listers
Generating informers for samplecontroller:v1alpha1 at k8s.io/sample-controller/pkg/generated/informers
$ ls pkg/apis/samplecontroller/v1alpha1/zz_generated.deepcopy.go
ls: pkg/apis/samplecontroller/v1alpha1/zz_generated.deepcopy.go: No such file or directory

Am I misunderstanding how this is supposed to work?

farah commented 4 years ago

@dims @sttts Hey guys, any idea on how to solve this? It's a bit confusing for beginners.

k8s.io$ git clone https://github.com/kubernetes/sample-controller
k8s.io$ cd sample-controller
sample-controller$ ./hack/update-codegen.sh
bash: ../code-generator/generate-groups.sh: No such file or directory

thanks in advance.

max-rocket-internet commented 4 years ago

TBH I think this code generation thing adds more headache/complexity than it solves, especially when you're just beginning to write a controller using this repo as a start.

farah commented 4 years ago

hey @max-rocket-internet, just realised that the fix is actually in the read me. Run go mod vendor and you should be good.

caboteria commented 4 years ago

Here's what I've figured out from a couple of hours of faffing around with this:

The first problem is "how do you run generate-groups.sh"? There are two answers. One was pointed out by @farah : populate the vendor directory. The other is to clone the code-generator project so that it's a sibling of sample-controller (which is where it lives in the kubernetes/kubernetes repo). You can look in hack/update-codegen.sh and you'll see that the script looks in the vendor directory and then falls back to ../code-generator.

The second problem is "where does generate-groups.sh put the files that it generates"? Remember how this repo is a mirror of a small part of kubernetes/kubernetes? Well, generate-groups.sh assumes and requires that everything is laid out exactly like it is in the kubernetes project. In that project, sample-controller is in k8s.io/sample-controller, i.e. {parent}/{project}. You'll see in generate-groups.sh that it goes up three directories, then uses the module path as a directory path which means that it doesn't work properly if your project's module isn't {parent}/{project}. You can hack update-codegen.sh to use a different output-base but you might still end up having to move the files after they're generated. I did.

This smells like it was written before Go modules and never updated to work with modules since it just so happens to work with the directory structure in the kubernetes/kubernetes project.

HTH

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

ferama commented 3 years ago

Did you find any solution? The go mod vendor doesn't work for me. This is what I did:

$ git clone https://github.com/kubernetes/sample-controller
$ cd sample-controller
$ go mod vendor
$ ./hack/update-codegen.sh

The script generates a bunch of dirs and filess inside my home

k8s.io/
k8s.io/sample-controller
k8s.io/sample-controller/pkg
k8s.io/sample-controller/pkg/generated
k8s.io/sample-controller/pkg/generated/clientset
k8s.io/sample-controller/pkg/generated/clientset/versioned
k8s.io/sample-controller/pkg/generated/clientset/versioned/scheme
k8s.io/sample-controller/pkg/generated/clientset/versioned/scheme/doc.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/scheme/register.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/typed
k8s.io/sample-controller/pkg/generated/clientset/versioned/typed/samplecontroller
k8s.io/sample-controller/pkg/generated/clientset/versioned/typed/samplecontroller/v1alpha1
k8s.io/sample-controller/pkg/generated/clientset/versioned/typed/samplecontroller/v1alpha1/generated_expansion.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/typed/samplecontroller/v1alpha1/samplecontroller_client.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/typed/samplecontroller/v1alpha1/fake
k8s.io/sample-controller/pkg/generated/clientset/versioned/typed/samplecontroller/v1alpha1/fake/doc.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/typed/samplecontroller/v1alpha1/fake/fake_samplecontroller_client.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/typed/samplecontroller/v1alpha1/doc.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/fake
k8s.io/sample-controller/pkg/generated/clientset/versioned/fake/clientset_generated.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/fake/doc.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/fake/register.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/clientset.go
k8s.io/sample-controller/pkg/generated/clientset/versioned/doc.go

but all the files there are empty

max-rocket-internet commented 3 years ago

I switched to kubebuilder, I think it's better.

fejta-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 3 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close

k8s-ci-robot commented 3 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes/sample-controller/issues/73#issuecomment-769711267): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-testing, kubernetes/test-infra and/or [fejta](https://github.com/fejta). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
vadasambar commented 2 years ago

I edited my update-codegen script like this:

bash "${CODEGEN_PKG}"/generate-groups.sh "deepcopy,client,informer,lister" \
  pkg/generated pkg/apis \
  mycontroller:v1alpha1 \
  --output-base "$(dirname "${BASH_SOURCE[0]}")/../" \
  --go-header-file "${SCRIPT_ROOT}"/hack/boilerplate.go.txt

This generates the files under the parent directory's pkg folder but all of the files are empty as mentioned in this comment. Wonder if anyone found a solution for the empty files.

vadasambar commented 2 years ago

/remove-lifecycle rotten

vadasambar commented 2 years ago

/reopen

k8s-ci-robot commented 2 years ago

@vadasambar: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to [this](https://github.com/kubernetes/sample-controller/issues/73#issuecomment-1200712194): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
max-rocket-internet commented 2 years ago

/reopen

k8s-ci-robot commented 2 years ago

@max-rocket-internet: Reopened this issue.

In response to [this](https://github.com/kubernetes/sample-controller/issues/73#issuecomment-1200814204): >/reopen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
max-rocket-internet commented 2 years ago

Reopened for @vadasambar but I moved on from this topic a long time ago 🙂

vadasambar commented 2 years ago

I can reproduce the same problem when I try running the update-codegen.sh script against the original sample-controller inside the kubernetes repo. 

suraj@suraj:~/sandbox/kubernetes/staging/src/k8s.io/sample-controller$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
suraj@suraj:~/sandbox/kubernetes/staging/src/k8s.io/sample-controller$ tree pkg ``` pkg ├── apis │ └── samplecontroller │ ├── register.go │ └── v1alpha1 │ ├── doc.go │ ├── register.go │ ├── types.go │ └── zz_generated.deepcopy.go ├── generated │ ├── clientset │ │ └── versioned │ │ ├── clientset.go │ │ ├── doc.go │ │ ├── fake │ │ │ ├── clientset_generated.go │ │ │ ├── doc.go │ │ │ └── register.go │ │ ├── scheme │ │ │ ├── doc.go │ │ │ └── register.go │ │ └── typed │ │ └── samplecontroller │ │ └── v1alpha1 │ │ ├── doc.go │ │ ├── fake │ │ │ ├── doc.go │ │ │ ├── fake_foo.go │ │ │ └── fake_samplecontroller_client.go │ │ ├── foo.go │ │ ├── generated_expansion.go │ │ └── samplecontroller_client.go │ ├── informers │ │ └── externalversions │ │ ├── factory.go │ │ ├── generic.go │ │ ├── internalinterfaces │ │ │ └── factory_interfaces.go │ │ └── samplecontroller │ │ ├── interface.go │ │ └── v1alpha1 │ │ ├── foo.go │ │ └── interface.go │ └── listers │ └── samplecontroller │ └── v1alpha1 │ ├── expansion_generated.go │ └── foo.go └── signals ├── signal.go ├── signal_posix.go └── signal_windows.go 21 directories, 30 files ```
suraj@suraj:~/sandbox/kubernetes/staging/src/k8s.io/sample-controller$ cat pkg/generated/clientset/versioned/clientset.go ```go /* Copyright The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // Code generated by client-gen. DO NOT EDIT. package versioned import ( "fmt" "net/http" discovery "k8s.io/client-go/discovery" rest "k8s.io/client-go/rest" flowcontrol "k8s.io/client-go/util/flowcontrol" samplecontrollerv1alpha1 "k8s.io/sample-controller/pkg/generated/clientset/versioned/typed/samplecontroller/v1alpha1" ) type Interface interface { Discovery() discovery.DiscoveryInterface SamplecontrollerV1alpha1() samplecontrollerv1alpha1.SamplecontrollerV1alpha1Interface } // Clientset contains the clients for groups. Each group has exactly one // version included in a Clientset. type Clientset struct { *discovery.DiscoveryClient samplecontrollerV1alpha1 *samplecontrollerv1alpha1.SamplecontrollerV1alpha1Client } // SamplecontrollerV1alpha1 retrieves the SamplecontrollerV1alpha1Client func (c *Clientset) SamplecontrollerV1alpha1() samplecontrollerv1alpha1.SamplecontrollerV1alpha1Interface { return c.samplecontrollerV1alpha1 } // Discovery retrieves the DiscoveryClient func (c *Clientset) Discovery() discovery.DiscoveryInterface { if c == nil { return nil } return c.DiscoveryClient } // NewForConfig creates a new Clientset for the given config. // If config's RateLimiter is not set and QPS and Burst are acceptable, // NewForConfig will generate a rate-limiter in configShallowCopy. // NewForConfig is equivalent to NewForConfigAndClient(c, httpClient), // where httpClient was generated with rest.HTTPClientFor(c). func NewForConfig(c *rest.Config) (*Clientset, error) { configShallowCopy := *c if configShallowCopy.UserAgent == "" { configShallowCopy.UserAgent = rest.DefaultKubernetesUserAgent() } // share the transport between all clients httpClient, err := rest.HTTPClientFor(&configShallowCopy) if err != nil { return nil, err } return NewForConfigAndClient(&configShallowCopy, httpClient) } // NewForConfigAndClient creates a new Clientset for the given config and http client. // Note the http client provided takes precedence over the configured transport values. // If config's RateLimiter is not set and QPS and Burst are acceptable, // NewForConfigAndClient will generate a rate-limiter in configShallowCopy. func NewForConfigAndClient(c *rest.Config, httpClient *http.Client) (*Clientset, error) { configShallowCopy := *c if configShallowCopy.RateLimiter == nil && configShallowCopy.QPS > 0 { if configShallowCopy.Burst <= 0 { return nil, fmt.Errorf("burst is required to be greater than 0 when RateLimiter is not set and QPS is set to greater than 0") } configShallowCopy.RateLimiter = flowcontrol.NewTokenBucketRateLimiter(configShallowCopy.QPS, configShallowCopy.Burst) } var cs Clientset var err error cs.samplecontrollerV1alpha1, err = samplecontrollerv1alpha1.NewForConfigAndClient(&configShallowCopy, httpClient) if err != nil { return nil, err } cs.DiscoveryClient, err = discovery.NewDiscoveryClientForConfigAndClient(&configShallowCopy, httpClient) if err != nil { return nil, err } return &cs, nil } // NewForConfigOrDie creates a new Clientset for the given config and // panics if there is an error in the config. func NewForConfigOrDie(c *rest.Config) *Clientset { cs, err := NewForConfig(c) if err != nil { panic(err) } return cs } // New creates a new Clientset for the given RESTClient. func New(c rest.Interface) *Clientset { var cs Clientset cs.samplecontrollerV1alpha1 = samplecontrollerv1alpha1.New(c) cs.DiscoveryClient = discovery.NewDiscoveryClient(c) return &cs } ``` 
suraj@suraj:~/sandbox/kubernetes/staging/src/k8s.io/sample-controller$ sudo rm -R pkg/generated pkg/apis
suraj@suraj:~/sandbox/kubernetes/staging/src/k8s.io/sample-controller$ tree pkg ``` pkg └── signals ├── signal.go ├── signal_posix.go └── signal_windows.go 1 directory, 3 files ``` 
suraj@suraj:~/sandbox/kubernetes/staging/src/k8s.io/sample-controller$ ./hack/update-codegen.sh 
Generating deepcopy funcs
Generating clientset for samplecontroller:v1alpha1 at k8s.io/sample-controller/pkg/generated/clientset
Generating listers for samplecontroller:v1alpha1 at k8s.io/sample-controller/pkg/generated/listers
Generating informers for samplecontroller:v1alpha1 at k8s.io/sample-controller/pkg/generated/informers
suraj@suraj:~/sandbox/kubernetes/staging/src/k8s.io/sample-controller$ tree pkg ``` pkg ├── generated │ └── clientset │ └── versioned │ ├── clientset.go │ ├── doc.go │ ├── fake │ │ ├── clientset_generated.go │ │ ├── doc.go │ │ └── register.go │ ├── scheme │ │ ├── doc.go │ │ └── register.go │ └── typed │ └── samplecontroller │ └── v1alpha1 │ ├── doc.go │ ├── fake │ │ ├── doc.go │ │ └── fake_samplecontroller_client.go │ ├── generated_expansion.go │ └── samplecontroller_client.go └── signals ├── signal.go ├── signal_posix.go └── signal_windows.go 10 directories, 15 files ```
suraj@suraj:~/sandbox/kubernetes/staging/src/k8s.io/sample-controller$ cat pkg/generated/clientset/versioned/clientset.go ```go /* Copyright The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ // Code generated by client-gen. DO NOT EDIT. package versioned ```
vadasambar commented 2 years ago

Started a thread around the problem in Kubernetes slack here

vadasambar commented 2 years ago

This works for me:

#!/usr/bin/env bash

# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -o errexit
set -o nounset
set -o pipefail

SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
CODEGEN_PKG=${CODEGEN_PKG:-$(cd "${SCRIPT_ROOT}"; ls -d -1 ./vendor/k8s.io/code-generator 2>/dev/null || echo ../code-generator)}

# Pre-requisites
# 1. this script needs ./pkg/apis/samplecontroller/v1alpha1 to generate everything else (generated files will be empty otherwise)
# 2. changing --output-base to something else doesn't work
# 
# Only the following things can be auto-generated:
# 1. ./pkg/apis/samplecontroller/v1alpha1/zz_generated.deepcopy.go
# 2. everything under ./pkg/apis/generated
bash "${CODEGEN_PKG}"/generate-groups.sh "deepcopy,client,informer,lister" \
  k8s.io/sample-controller/pkg/generated k8s.io/sample-controller/pkg/apis \
  samplecontroller:v1alpha1 \
  --output-base "$(dirname "${BASH_SOURCE[0]}")/../../.." \
  --go-header-file "${SCRIPT_ROOT}"/hack/boilerplate.go.txt

# This block:
# output_base: the directory outside the repo where auto-generated files are created
# 1. removes existing auto-generated files
# 2. copies over the auto-generated files from the output_base to pkg/apis and pkg/generated
# 3. removes output_base
rm -R ./pkg/apis/samplecontroller/v1alpha1/zz_generated.deepcopy.go ./pkg/generated -f
mv "$(dirname "${BASH_SOURCE[0]}")/../../../k8s.io/sample-controller/pkg/apis/samplecontroller/v1alpha1/zz_generated.deepcopy.go" $SCRIPT_ROOT/pkg/apis/samplecontroller/v1alpha1 -f
mv "$(dirname "${BASH_SOURCE[0]}")/../../../k8s.io/sample-controller/pkg/generated" $SCRIPT_ROOT/pkg
rm -R "$(dirname "${BASH_SOURCE[0]}")/../../../k8s.io"

# To use your own boilerplate text append:
#   --go-header-file "${SCRIPT_ROOT}"/hack/custom-boilerplate.go.txt
k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 1 year ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes/sample-controller/issues/73#issuecomment-1373162388): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.