Problem with codegen on Windows

[xkisu]

I downloaded the repo, modified the api files and codegen script to suit my needs and tried to run update-codegen.sh from Git Bash, but it failed with these errors.

Keith@DESKTOP-021AQFL MINGW64 ~/go/src/gamepod.cc/agent (master)
$ ./hack/update-codegen.sh
Generating deepcopy funcs
Generating clientset for gamepod:v1alpha1 at gamepod.cc/agent/pkg/client/clientse                          t
W1223 18:13:29.376607   33732 import_tracker.go:45] Warning: backslash used in i                          mport path 'gamepod.cc\agent\pkg\client\clientset\versioned\scheme', this is uns                          upported.
F1223 18:13:29.441608   33732 main.go:64] Error: Failed executing generator: som                          e packages had errors:
errors in package "gamepod.cc\\agent\\pkg\\client\\clientset\\versioned":
unable to format file "..\\..\\gamepod.cc\\agent\\pkg\\client\\clientset\\versio                          ned\\clientset.go" (20:34: unknown escape sequence (and 3 more errors)).

errors in package "gamepod.cc\\agent\\pkg\\client\\clientset\\versioned\\fake":
unable to format file "..\\..\\gamepod.cc\\agent\\pkg\\client\\clientset\\versio                          ned\\fake\\clientset_generated.go" (20:34: unknown escape sequence (and 10 more                           errors)).

errors in package "gamepod.cc\\agent\\pkg\\client\\clientset\\versioned\\typed\\                          gamepod\\v1alpha1":
unable to format file "..\\..\\gamepod.cc\\agent\\pkg\\client\\clientset\\versio                          ned\\typed\\gamepod\\v1alpha1\\gameserver.go" (22:11: illegal character U+005C '\                          ' (and 10 more errors)).
unable to format file "..\\..\\gamepod.cc\\agent\\pkg\\client\\clientset\\versio                          ned\\typed\\gamepod\\v1alpha1\\gamepod_client.go" (17:2: invalid import path: "ga                          mepod.cc\\agent\\pkg\\client\\clientset\\versioned\\scheme").

errors in package "gamepod.cc\\agent\\pkg\\client\\clientset\\versioned\\typed\\                          gamepod\\v1alpha1\\fake":
unable to format file "..\\..\\gamepod.cc\\agent\\pkg\\client\\clientset\\versio                          ned\\typed\\gamepod\\v1alpha1\\fake\\fake_gamepod_client.go" (19:28: unknown esca                          pe sequence (and 3 more errors)).

Where gamepod is my api name and gameserver is my CRD. Upon looking in some of the files in the error message I found out that it was using backslashes in import paths for files it generated:

// pkg/client/clientset/versioned/clientset.go

...

package versioned

import (
    flowcontrol "k8s.io/client-go/util/flowcontrol"
    stablev1alpha1 "gamepod.cc\agent\pkg\client\clientset\versioned\typed\gamepod\v1alpha1"
    discovery "k8s.io/client-go/discovery"
    rest "k8s.io/client-go/rest"
)

...

However when I run update-codegen.sh with the same code on a Linux VPS it runs completely fine and generates the correct import paths so it appears this is a problem caused by the scripts not using the correct path separator under Git Bash/MSYS2.

I assume this is less of an issue with sample-controller and probably an issue with kubernetes/code-generator or kubernetes/gengo but I'm not sure where to look so I assume this is the best place to start.

Here are the most relevant files:

// hack/update-codegen.sh

set -o errexit
set -o nounset
set -o pipefail
SCRIPT_ROOT=$(dirname ${BASH_SOURCE})/..
CODEGEN_PKG=${CODEGEN_PKG:-$(cd ${SCRIPT_ROOT}; ls -d -1 ./vendor/k8s.io/code-generator 2>/dev/null || echo ../code-generator)}
# generate the code with:
# --output-base    because this script should also be able to run inside the vendor dir of
#                  k8s.io/kubernetes. The output-base is needed for the generators to output into the vendor dir
#                  instead of the $GOPATH directly. For normal projects this can be dropped.
${CODEGEN_PKG}/generate-groups.sh "deepcopy,client,informer,lister" \
  gamepod.cc/agent/pkg/client gamepod.cc/agent/pkg/apis \
  gamepod:v1alpha1 \
  --output-base "$(dirname ${BASH_SOURCE})/../../.." \
  --go-header-file ${SCRIPT_ROOT}/hack/boilerplate.go.txt

// pkg/apis/gamepod/v1alpa1/types.go

package v1alpa1

import (
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// GameServer is a specification for a Game Server resource
type GameServer struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`
    Spec              GameServerSpec   `json:"spec"`
    Status            GameServerStatus `json:"status"`
}

// GameServerSpec is the spec for a GameServer resource
type GameServerSpec struct {
    Game GameServerGame `json:"game"`
}

// GameServerGame is the spec for the Game key of the GameServer resource spec
type GameServerGame struct {
    Name    string `json:"name"`
    Version string `json:"version"`
}

// GameServerStatus is the status for a GameServer resource
type GameServerStatus struct {
    Created bool `json:"installed"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// GameServerList is a list of GameServer resources
type GameServerList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata"`
    Items           []GameServer `json:"items"`
}

[xkisu]

This error also occurs if I just clone the sample-controller repository and run update-codegen.sh without modifying the repository at all.

[xkisu]

Upon further inspection certain generated files get the import path separator correct and some don't, specifically these ones don't:

pkg/client/clientset/versioned/fake/clientset_generated.go
pkg/client/clientset/versioned/typed/samplecontroller/v1alpha1/fake/fake_samplecontroller_client.go
pkg/client/clientset/versioned/typed/samplecontroller/v1alpha1/foo.go
pkg/client/clientset/versioned/typed/samplecontroller/v1alpha1/samplecontroller_client.go

The informers and listers get the imports correct, and most of the clientset except for those specific files above.

[xkisu]

I've tracked this down to an issue with code-generator's client-gen using path/filepath#Join() for generating import paths. filepath#Join uses the OS's file separator and that's why it's using baskslashes, it's usage in some places should be replaced with path#Join which uses forward slashes.

I'll open a pull request for it when I get a chance.

[chrislinan]

Hi @xkisu

I met the same issue with you. You mentioned that you will open a pull request for this issue, is there any update?

Thanks a lot.