Cannot install with git-for-windows: config path broken

[WolfByttner]

Kubeprod will not correctly resolve the path to kubeprod-autogen.json. When I try, I get various strange error messages or weird file paths. I have attempted all file format paths that I can think of (see below for outputs).

System Info

• kubeprod version BKPR v1.6.1
• kubecfg version 1c66c525e183d8078fc05be97bae332d414c4af4 (built from source)
• go version go1.16 windows/amd64
• Windows 10 Version 10.0.19042 Build 19042
• Git for windows version 2.24.0.2

Steps to reproduce

1. Setup a kubernetes cluster (I use GKE).
2. Build kubecfg for windows.
3. Add all required binaries to your path so Windows can see them.
4. Run the test script in git-for-windows (commonly known as "git bash") with your credentials and a path to kubeprod-autogen.json

Test script:

gcloud auth activate-service-account my-serviceaccount@my-project.iam.gserviceaccount.com --key-file=./gke-credentials.json

gcloud config set account my-serviceaccount@my-project.iam.gserviceaccount.com

export GCLOUD_ZONE="my-zone"

export BKPR_DNS_ZONE="my-cluster.my-domain.com"
export GCLOUD_USER="$(gcloud info --format='value(config.account)')"
export GCLOUD_PROJECT="my-project"
export GCLOUD_AUTHZ_DOMAIN="my-domain.com"
export GCLOUD_K8S_CLUSTER="my-cluster-name"

kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole=cluster-admin \
  --user=${GCLOUD_USER}

export GCLOUD_OAUTH_CLIENT_KEY="XXXXXXXXXXXXXXXXXXX"
export GCLOUD_OAUTH_CLIENT_SECRET="XXXXXXXXXXXXXXXXXXXXXXXXX"

kubeprod install gke \
  --email "${GCLOUD_USER}" \
  --dns-zone "${BKPR_DNS_ZONE}" \
  --project "${GCLOUD_PROJECT}" \
  --oauth-client-id "${GCLOUD_OAUTH_CLIENT_KEY}" \
  --oauth-client-secret "${GCLOUD_OAUTH_CLIENT_SECRET}" \
  --authz-domain "${GCLOUD_AUTHZ_DOMAIN}" \
  --kubeconfig "${KUBE_CONFIG}" \
  --config ${MY_FILEPATH}

Let us start with no --config flag. You get the following output.

$ ./install-kubeprod.sh
Installing platform gke
Error: open /C:\Users\wolfb\test/kubeprod-autogen.json: The filename, directory name, or volume label syntax is incorrect.

With git bash's way of writing things; "/c/Users/wolfb/test/my-manifest-file.json"; you instead get the error:

$ ./install-kubeprod.sh
Error: platform config path must be a file:// URL

With the (canonical) filepath "C:\\Users\\wolfb\\test\\my-kubeprod-autogen.json" you get the same error again:

$ ./install-kubeprod.sh
Error: platform config path must be a file:// URL

The path "file:///C:/Users/wolfb/test/my-kubeprod-autogen.json" can be visited in a browser. It seems like kubeprod expects this format.

$ ./install-kubeprod.sh
Installing platform gke
Error: open /C:/Users/wolfb/test/my-kubeprod-autogen.json: The filename, directory name, or volume label syntax is incorrect.

Removing the colon, "file:///C:/Users/wolfb/test/my-kubeprod-autogen.json":

$ ./install-kubeprod.sh
Error: parse "file\\\\\\C;C:\\Programs\\Git\\Users\\wolfb\\test\\my-kubeprod-autogen.json": first path segment in URL cannot contain colon

Removing one of the slashes (your web browser will add a slash when you paste the link) we get:

$ ./install-kubeprod.sh
Error: parse "file\\\\C;C:\\Programs\\Git\\Users\\wolfb\\test\\my-kubeprod-autogen.json": first path segment in URL cannot contain colon

With lowercase "c" (a Hail Mary approach) "file//c:/Users/wolfb/test/my-kubeprod-autogen.json" we get:

$ ./install-kubeprod.sh
Error: parse "file\\\\c;C:\\Programs\\Git\\Users\\wolfb\\test\\my-kubeprod-autogen.json": first path segment in URL cannot contain colon

Debugging

I am working on a test script, however I have yet to find the root cause of the issue: Below is a work in progress:

package main

// We then want to use the fmt package
// which features a `print` function - Println

import (
    "fmt"
    "net/url"
    "os"
)

// We then need to define our main function.
// Think of this as the entry point to our Go
// program
func main() {
    cwd, err := os.Getwd()
    fmt.Println("cwd", cwd, err)

    fmt.Println("rawURL", &url.URL{Scheme: "file", Path: cwd}, nil)
    if cwd[len(cwd)-1] != '/' {
            cwd = cwd + "/"
    }

    cwdURL := &url.URL{Scheme: "file", Path: cwd}

    fmt.Println("cwdURL", cwdURL)

    platformConfigBase := "C:\\Users\\wolfb\\test\\my-manifest-file.json"

    platformConfigURL, err := cwdURL.Parse(platformConfigBase)

    fmt.Println("platformConfigURL", platformConfigURL, err)

    fmt.Println("platformConfigURL.Scheme", platformConfigURL.Scheme)
    if platformConfigURL.Scheme != "file" {
        fmt.Println(fmt.Errorf("platform config path must be a file:// URL"))
    }

    platformConfigBase2 := "file:///C:/Users/wolfb/test/my-manifest-file.json"

    platformConfigURL2, err := cwdURL.Parse(platformConfigBase2)

    fmt.Println("platformConfigURL2", platformConfigURL2, err)

    fmt.Println("platformConfigURL2.Scheme", platformConfigURL2.Scheme)
    if platformConfigURL2.Scheme != "file" {
        fmt.Println(fmt.Errorf("platform config path must be a file:// URL"))
    }
}

We get the following output:

$ go build testdir.go ; ./testdir
cwd C:\Users\wolfb\test <nil>
rawURL file://C:%5CUsers%5Cwolfb%5Ctest <nil>
cwdURL file://C:%5CUsers%5Cwolfb%5Ctest/
platformConfigURL c:\Users\wolfb\test\my-manifest-file.json <nil>
platformConfigURL.Scheme c
platform config path must be a file:// URL
platformConfigURL2 file:///C:/Users/wolfb/test/my-manifest-file.json <nil>
platformConfigURL2.Scheme file

You can see how go successfully converts the file to a URL and back to a legible string. However, at some point after that, the string is mangled beyond recognition.

[WolfByttner]

After some debugging I found the cause of the problem. The offending line is 112 in kubeprod/cmd/install.go.

If I hardcode the string value, the program loads the file just fine. c.PlatformConfigPath = "C:/Users/wolfb/test/my-kubeprod-autogen.json" //platformConfigURL.Path

Looking in the debugger, c.PlatformConfigPath takes the value /C:/Users/wolfb/test/my-kubeprod-autogen.json - the leading slash breaks the Windows path recognition.

I understand the intention here, but I doubt that the assumption that file paths are URLs is universally valid. This problem looks similar to a core language issue that was identified (#https://github.com/golang/go/issues/13276).

[WolfByttner]

I found a silly workaround. If on windows, strip the first character. This likely works if the user has specified the full file path, since Go cannot resolve the scheme without the file:// prefix (see #https://github.com/golang/go/issues/13276). Thus the user will likely have a leading slash, that has to be removed.

        if platformConfigURL.Scheme != "file" {
        return nil, fmt.Errorf("platform config path must be a file:// URL")
    }
    c.PlatformConfigPath = platformConfigURL.Path
    if runtime.GOOS == "windows" {
        c.PlatformConfigPath = c.PlatformConfigPath[1:]
    }

However, I'd struggle to call this an optimal solution. Have we hit on a deeper assumption here - one that is ultimately not correct? And how can we go about resolving this issue in the codebase?

[WolfByttner]

There is an additional downstream error with kubeprod-manifest.jsonnet. This time the URL is used directly with undesirable results.

Updating platform gke
Error: Could not get candidate URLs for when importing file://C:%5CUsers%5Cwolfb%5Cgit%5Ckube-prod-runtime/kubeprod-manifest.jsonnet (imported from file://C:%5CUsers%5Cwolfb%5Cgit%5Ckube-prod-runtime/): Import path "file://C:%5CUsers%5Cwolfb%5Cgit%5Ckube-prod-runtime/kubeprod-manifest.jsonnet" is not valid

The root cause here (after applying our URL slicing) is that backslashes get encoded into %5C rather than a more helpful forward slash.

On a positive note - the previous solution works whether one specifies a config explicitly or not. The file name does look strange (C:\\Users\\wolfb\\git\\test/kubeprod-autogen.json) but the os module resolves the filename correctly.