Go Build Fails on Windows With Long Paths

[dedimarco]

Build error on Windows system, due to long filenames (in V1 directory in particular)

See Go Issues 18468

Resolved by rename all files, but not easy, too many files.

[antihax]

Thanks. Unfortunately this is a Go compiler issue and not an issue with ESI or goesi.

I personally moved my dev environment to a linux VM as a workaround until the Go team resolves the issue.

You may want to place your modified version of goesi into /vendor since this repository automatically updates if CCP makes a spec change.

[deveyus]

I had this problem, after much work I found that you can use Group Policy to set "Enable win32 long paths", and on home versions you can use registry editor to change LongPathsEnabled to 1, to enable longer path names.

[jpastoor]

The longPaths registry change didn't work for me, so I wrote a little hack to rename the filenames. (Won't bother me much since I'm using an IDE to browse the entities)

So no permanent solution or anything, but might help some so I am posting it here: https://gist.github.com/jpastoor/e2ab14e9126a74d4fa540c2f3cd66abd

[antihax]

CCP changed many of the model names which may assist with this issue. Can anyone confirm?

[dancodes]

I can confirm I can't build this on Windows at all, and go build won't help me even debug what the file is :(

[antihax]

I would recommend using a Linux VM. This is a limitation of Golang on Windows.

[regner]

This seems to be the root issue in go: https://github.com/golang/go/issues/18468

Basically when go compiles a package it builds a syscall with all the file paths, and if that syscall is over 32k characters 📛 🔥 RIP.

[regner]

I don't know what is involved in that syscall for the compile, but JUST the filenames in the esi folder is 38,750 characters.

• Changing models_ -> m_ brings it down to 36,030.
• Changing delete_ -> d_, get_ -> g_, and post_ -> p_ brings it down to 34,612
• Changing character -> char brings it down to 32,347
• Changing corporation -> corp brings it down to 30,590

I did a quick test with the following script:

package main

import (
    "fmt"
    "os"
    "path/filepath"
    "strings"
)

func main() {
    filepath.Walk("vendor/github.com/antihax/goesi/esi", derpify)
}

func derpify(path string, f os.FileInfo, err error) error {
    mapping := map[string]string{
        "models_":      "m_",
        "delete_":      "d_",
        "get_":         "g_",
        "post_":        "p_",
        "_character":   "_char",
        "_corporation": "_corp",
    }

    if filepath.Ext(path) == ".go" {
        fmt.Println(path)
        newpath := path

        for k, v := range mapping {
            newpath = strings.Replace(newpath, k, v, -1)
        }

        os.Rename(path, newpath)
    }
    return nil
}

Ran that and goesi compiles on Windows.

[regner]

@antihax are you open to working around this for goesi? If you are I see a few possible options:

1. Something like the above script is added to the https://github.com/antihax/swagger-esi-goclient repo since thats where the code is generated. This has the advantage that if people look at files they still make some kind of sense. There are varying levels of aggressiveness that this can be applied with that will change how much sense the files still make vs how future proof it is.
2. Saying "fuck it, no one needs sane file names" and just rename everything with an incrementing number. This has the advantage of being far more future proof, but you do lose the ability to read through the files with any level of sanity and doing a git history of a file becomes harder. Any time a new model is added everything after that in the list gets a new number.
3. Not wanting to touch the filenames in the repo but including some like the above script that people on Windows can run. This paired with a note in the README would help direct those on Windows to solve their issue.

Thoughts?

[antihax]

This should be fine to run post process. Sane names are not really necessary. If you can write something and send a PR i will give it a shot until they resolve the issue with the compiler.

[regner]

Just for clarity, do you prefer option 1 or 2? I will try and get a PR up tonight or tomorrow into the other repo.

[antihax]

Try 1. If you run into problem still, could change to 2.

[regner]

Sounds good.

[HakShak]

I have a hilarious workaround that is more of a toy implementation than anything else if you have Docker for windows.

This cross compiles the goesi *.a files in a container, then litters the codebase with nobuild.go files to prevent the windows compiler from trying to help. Copy all the output over into your GOPATH and bam, a working windows build.

However, the windows_build image tag MUST match your installed windows version of golang or you will get mismatches when linking in the *.a files.

Again, this is a toy implementation that works and was mostly done for the lulz.

# Usage
# docker build -t goesi-build .
# docker create --name windows_amd64 goesi-build -
# docker cp windows_amd64:/go/pkg %GOPATH%
# docker cp windows_amd64:/go/src %GOPATH%

FROM golang:1.10.1 as windows_build 
# Download and build all .a for windows amd64
RUN GOOS=windows GOARCH=amd64 go get -v github.com/antihax/goesi/...
# Litter the codebase with nobuild.go which prevents the compiler from trying to make new .a files
WORKDIR /go/src/github.com/antihax/goesi
RUN echo "//go:binary-only-package\n\npackage goesi\n" >> nobuild.go
RUN echo "//go:binary-only-package\n\npackage esi\n" >> esi/nobuild.go
RUN echo "//go:binary-only-package\n\npackage optional\n" >> optional/nobuild.go

FROM scratch
WORKDIR /go/pkg/windows_amd64
COPY --from=windows_build /go/pkg/windows_amd64 .
WORKDIR /go/src/github.com/antihax/goesi
COPY --from=windows_build /go/src/github.com/antihax/goesi .

[antihax]

Will be fixed in next go release.

[regner]

@antihax have a link to the fix by any chance?

[antihax]

https://go-review.googlesource.com/c/go/+/110395

[regner]

Thanks for the link! :)