search

richardlehane / siegfried

signature-based file format identification
http://www.itforarchivists.com/siegfried
Apache License 2.0
224 stars 30 forks source link

glide: vendor missing pkgs #98

Closed sevein closed 7 years ago

sevein commented 7 years ago

While building the rpm package of siegfried for Archivematica @jhsimpson found a build error that I think it's caused by certain packages not being included in the vendor/ folder. It can be easily reproduced if you start with a clean GOPATH and you don't use go get so they're not automatically downloaded.

vendor/golang.org/x/text/cmd/gotext/main.go:29:2: cannot find package "golang.org/x/tools/go/buildutil" in any of:
    /Users/jesus/sfgo/src/github.com/richardlehane/siegfried/vendor/golang.org/x/tools/go/buildutil (vendor tree)
    /usr/local/Cellar/go/1.8rc3/libexec/src/golang.org/x/tools/go/buildutil (from $GOROOT)
    /Users/jesus/sfgo/src/golang.org/x/tools/go/buildutil (from $GOPATH)
vendor/golang.org/x/text/cmd/gotext/extract.go:23:2: cannot find package "golang.org/x/tools/go/loader" in any of:
    /Users/jesus/sfgo/src/github.com/richardlehane/siegfried/vendor/golang.org/x/tools/go/loader (vendor tree)
    /usr/local/Cellar/go/1.8rc3/libexec/src/golang.org/x/tools/go/loader (from $GOROOT)
    /Users/jesus/sfgo/src/golang.org/x/tools/go/loader (from $GOPATH)

Installing the following packages into vendor/ fixed the build:

golang.org/x/tools/go/buildutil
golang.org/x/tools/go/loader
golang.org/x/net/html
golang.org/x/net/html/atom
golang.org/x/net/websocket
golang.org/x/crypto/ssh/terminal

I used glide so my commit also includes the changes in glide.yaml and glide.lock.

sevein commented 7 years ago

I think that your CI setup probably didn't catch this issue before because go get github.com/richardlehane/siegfried/cmd/sf is downloading those dependencies not found in vendor/. I don't think that the go tool lets you disable that behaviour.

richardlehane commented 7 years ago

Hi Jesús I just changed the .travis.yml to use "go install" rather than "go get" to debug this. It seems to be building the sf exe fine without the extra deps (https://travis-ci.org/richardlehane/siegfried/builds/199448432) (predictably breaks on 1.4 and 1.5 due to vendoring).

I suspect these extra dependencies are only needed if you are trying to build all the main.go files within the siegfried repo, including any main.go files within the vendor directory e.g. golang.org/x/text/cmd/gotext/main.go. You really should just be building main.go files for sf and roy tools.

If you change your RPM script from go install -tags archivematica -v github.com/richardlehane/siegfried/... to go install -tags archivematica -v github.com/richardlehane/siegfried/cmd/... should fix I think?

sevein commented 7 years ago

including any main.go files within the vendor directory

Right, I missed that! Thank you, it worked. I'm relieved in that we didn't have to vendor +384,426 LoC, 😅