vendoring in a go.mod world

[mischief]

hi,

after our brief discussion over email, you vendored libauth. unfortunately, that didn't seem to convince go that it was actually vendored.

if you have go 1.11, and have a clean checkout of dgit, with NO $GOPATH set, and execute:

GOOS=plan9 go build -x

you will see that libauth is actually downloaded with a hg clone.

the context here is building dgit within a packaging system on 9front where network access is not allowed during the build step.

as i've been away from the go world for some time, i'm not familiar with any of this newfangled mod stuff, but i expect vendoring to work if there's a vendor directory.

any ideas?

[driusan]

I honestly haven't been following the Go community very actively recently either. I know rsc originally wanted to kill vendoring entirely, but was persuaded to step down slightly and make vendoring work only at the top level package (at least that's what the faq says).. that's actually why I didn't respond to your email and haven't double checked for other dependencies yet: I don't really know the implications or future of vendoring any more and don't want to commit to anything until I do.

It looks like there's also a -mod mode build flag now in the FAQ. Maybe try using that and see if it helps? (If it doesn't and I didn't do something stupid like spell your name wrong, I think it's a bug in Go since dgit is a top level package)

[sirnewton01]

As a workaround, I wonder if putting libauth in the place where go build would normally fetch it would work. I'm assuming that dgit is being cloned and put somewhere during a step where the network is available and so libauth might be gettable the same way at the same time.

I've been wondering about a packaging system on top of 9front for Go tools myself. I am still trying to figure out how to deal with the Go must be built with previous version of Go problem mixed with the trouble with trying to compile the last C source version of Go on Plan 9.

[mischief]

@sirnewton01 we have solved that via the normal method. feel free to contact me via email, freenode or the 9fans discord for further information.

[driusan]

So did go build -x -mod=vendor work?

[mischief]

i just tried it -

cpu% go build -x -mod vendor
WORK=/tmp/go-build713872532
build bitbucket.org/mischief/libauth: cannot find module for path bitbucket.org/mischief/libauth
cpu% lc
.gitignore          git/                official-git/
.travis.yml         go-get-tests.sh*    status.txt
CONTRIBUTING.md     go.mod              vendor/
LICENSE             gopath/             zlib/
README.md           issues/
cmd/                main.go
cpu% du vendor
14  vendor/bitbucket.org/mischief/libauth
14  vendor/bitbucket.org/mischief
14  vendor/bitbucket.org
14  vendor
cpu% 

[driusan]

Can you pull the latest commit and try again? It looks like for >1.11 you might need to manage the directory using go mod vendor and can't just manually maintain it.

[mischief]

@driusan ok - it's working now, even with no network access.

cpu% git
Usage: git [global options] subcommand [options] 

Global options:

  -C string
        chdir before starting git
  -git-dir string
        specify the repository of git
  -super-prefix string
        useless option used internally by git test suite
  -work-tree string
        specify the working directory of git

this will be available in the 9front ports system.

as an aside, it would be great to tag a release, even it's it's 0.0.1 or whatever.

fixed in 8451bc2d419af47b5f6ab4c920e2ac5bbeece8e3.

[driusan]

The main reason I'm hesitant to tag a release is because I don't really know what effect that will have on go get when combined with go modules, and I don't want to lose the behaviour where it'll follow HEAD

[mischief]

that's ok, the main reason i ask is because i added manifest generation and verification to ports very recently. the details are still being sussed out, but right now there's no opt-out (like gentoo's "live" ebuilds"), so if a port tracks an ever-changing tarball like a git repo's master (such as dgit), you'll get hash verification failure any time upstream changes. a release would let me be lazy and delay implementing opt-out ;-) but i can also pin a commit hash instead of using master.

[driusan]

Would it help if I tagged a non-semanticly versioned release named something like "vbootstrap" that vgo wouldn't consider a valid version number?