x/vuln: Does not use the correct GOPATH: Index(): mkdir /pkg: permission denied

evanj commented 1 year ago

Summary: I have no Go-related environment variables set, other than putting the Go tools in PATH. Go tools work, but govulncheck seems to try to cache files in the wrong place, and reports: govulncheck: vulncheck.Source: GetByModule("google.golang.org/protobuf"): httpSource.GetByModule("google.golang.org/protobuf"): Index(): mkdir /pkg: permission denied. The workaround is to explicitly setting the GOPATH environment variable: GOPATH=$(go env GOPATH) govulncheck ./... .

What version of Go are you using (`go version`)?

$ go version
go version go1.19.4 linux/amd64

Does this issue reproduce at the latest version of golang.org/x/vuln?

Yes. Specifically I ran:

$ go install golang.org/x/vuln/cmd/govulncheck@latest
go: downloading golang.org/x/vuln v0.0.0-20221217143601-785badac1c32
...

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ej/.cache/go-build"
GOENV="/home/ej/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ej/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ej/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/ej/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/ej/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.4"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2875784920=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I ran: govulncheck ./... in this repository: https://github.com/evanj/hacks

What did you expect to see?

I expected to see some valid output.

What did you see instead?

$ govulncheck ./...
govulncheck is an experimental tool. Share feedback at https://go.dev/s/govulncheck-feedback.

Scanning for dependencies with known vulnerabilities...
govulncheck: vulncheck.Source: GetByModule("google.golang.org/protobuf"): httpSource.GetByModule("google.golang.org/protobuf"): Index(): mkdir /pkg: permission denied

The problem is I have no Go related environment variables set, other than PATH:

$ env
SYSTEMD_EXEC_PID=2738
SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
SESSION_MANAGER=local/ejmeerkat:@/tmp/.ICE-unix/2093,unix/ejmeerkat:/tmp/.ICE-unix/2093
GNOME_TERMINAL_SCREEN=/org/gnome/Terminal/screen/b1bbcf68_156e_4d16_9d19_c1176d2c86e1
LANG=en_US.UTF-8
XDG_CURRENT_DESKTOP=ubuntu:GNOME
WAYLAND_DISPLAY=wayland-0
QT_IM_MODULE=ibus
DESKTOP_SESSION=ubuntu
USER=ej
XDG_MENU_PREFIX=gnome-
OLDPWD=/home/ej
HOME=/home/ej
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
SSH_AGENT_LAUNCHER=openssh
GTK_MODULES=gail:atk-bridge
_=/usr/bin/env
VTE_VERSION=7000
XDG_SESSION_DESKTOP=ubuntu
QT_ACCESSIBILITY=1
GNOME_DESKTOP_SESSION_ID=this-is-deprecated
GNOME_SETUP_DISPLAY=:1
LOGNAME=ej
GNOME_TERMINAL_SERVICE=:1.105
GNOME_SHELL_SESSION_MODE=ubuntu
PATH=/home/ej/google-cloud-sdk/bin:/home/ej/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/ej/go/bin:/home/ej/go/bin
XMODIFIERS=@im=ibus
SHELL=/bin/zsh
XDG_SESSION_TYPE=wayland
XDG_RUNTIME_DIR=/run/user/1000
XDG_DATA_DIRS=/usr/local/share/:/usr/share/:/var/lib/snapd/desktop
USERNAME=ej
COLORTERM=truecolor
XAUTHORITY=/run/user/1000/.mutter-Xwaylandauth.YRBWX1
PWD=/home/ej/hacks
XDG_SESSION_CLASS=user
TERM=xterm-256color
GDMSESSION=ubuntu
DISPLAY=:0
SHLVL=1
EDITOR=subl --wait
CLICOLOR=1
GREP_COLORS=1;34

To fix this error, I can explicitly set GOPATH, and run this tool with: GOPATH=$(go env GOPATH) govulncheck ./... . This works as expected.

My understanding is that Go tools should no longer require setting GOPATH.

dr2chase commented 1 year ago

@golang/vulndb

zpavlinovic commented 1 year ago

What does the Go standard tooling tell you? Does it produce a warning?

I am asking as I have a problem reproducing this.

gopherbot commented 1 year ago

Change https://go.dev/cl/459036 mentions this issue: internal: use go env to compute GOMODCACHE

evanj commented 1 year ago

The go command has not difficulty. E.g. go test ./... works fine.

Here is a complete reproduction script on a clean Ubuntu 20.04 LTS container:

Start the container with: docker run --rm -ti ubuntu:20.04

Script to run in the container:

# install curl
apt update
apt install --yes curl

# create a normal user account and run the rest of the commands as that user
useradd --create-home normaluser --shell /bin/bash
su --login normaluser

# install Go 1.19.4
curl --location https://go.dev/dl/go1.19.4.linux-amd64.tar.gz | tar xz
export PATH=$PATH:$HOME/go/bin

# get latest govulncheck
go install golang.org/x/vuln/cmd/govulncheck@latest

# create a program with a third-party dependency
mkdir exampleprogram
cd exampleprogram
go mod init example.com/exampleprogram
cat > main.go <<END_PROGRAM
package main

import (
  "fmt"

  "golang.org/x/text/unicode/runenames"
)

func main() {
  fmt.Printf("name for a: %s\n", runenames.Name('r'))
}
END_PROGRAM
go get golang.org/x/text/unicode/runenames

# run the program with Go tools (works)
go test ./...
go run .

# run govulncheck (does not work)
govulncheck ./...

# run govulncheck with explicit GOPATH (works)
echo "GOPATH?" $(go env GOPATH)
GOPATH=/home/normaluser/go govulncheck ./...

Example output I get when I run this (with a lot of output removed)

root@e15e7e6bb951:/# # install curl
root@e15e7e6bb951:/# apt update
Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
... LINES REMOVED ...
All packages are up to date.
root@e15e7e6bb951:/# apt install --yes curl
Reading package lists... Done
... LINES REMOVED ...
done.
root@e15e7e6bb951:/# # create a normal user account and run the rest of the commands as that user
root@e15e7e6bb951:/# useradd --create-home normaluser --shell /bin/bash
root@e15e7e6bb951:/# su --login normaluser
normaluser@e15e7e6bb951:~$ 
normaluser@e15e7e6bb951:~$ # install Go 1.19.4
normaluser@e15e7e6bb951:~$ curl --location https://go.dev/dl/go1.19.4.linux-amd64.tar.gz | tar xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    75  100    75    0     0     14      0  0:00:05  0:00:05 --:--:--    19
100  142M  100  142M    0     0  16.0M      0  0:00:08  0:00:08 --:--:-- 41.3M
normaluser@e15e7e6bb951:~$ export PATH=$PATH:$HOME/go/bin
normaluser@e15e7e6bb951:~$ 
normaluser@e15e7e6bb951:~$ # get latest govulncheck
normaluser@e15e7e6bb951:~$ go install golang.org/x/vuln/cmd/govulncheck@latest
go: downloading golang.org/x/vuln v0.0.0-20221217143601-785badac1c32
go: downloading golang.org/x/tools v0.4.1-0.20221217013628-b4dfc36097e2
go: downloading golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e
go: downloading golang.org/x/mod v0.7.0
go: downloading golang.org/x/sys v0.3.0
normaluser@e15e7e6bb951:~$ 
normaluser@e15e7e6bb951:~$ # create a program with a third-party dependency
normaluser@e15e7e6bb951:~$ mkdir exampleprogram
normaluser@e15e7e6bb951:~$ cd exampleprogram
normaluser@e15e7e6bb951:~/exampleprogram$ go mod init example.com/exampleprogram
go: creating new go.mod: module example.com/exampleprogram
normaluser@e15e7e6bb951:~/exampleprogram$ cat > main.go <<END_PROGRAM
... LINES REMOVED ...
> END_PROGRAM
normaluser@e15e7e6bb951:~/exampleprogram$ go get golang.org/x/text/unicode/runenames
go: downloading golang.org/x/text v0.5.0
go: added golang.org/x/text v0.5.0
normaluser@e15e7e6bb951:~/exampleprogram$ 
normaluser@e15e7e6bb951:~/exampleprogram$ # run the program with Go tools (works)
normaluser@e15e7e6bb951:~/exampleprogram$ go test ./...
?       example.com/exampleprogram  [no test files]
normaluser@e15e7e6bb951:~/exampleprogram$ go run .
name for a: LATIN SMALL LETTER R
normaluser@e15e7e6bb951:~/exampleprogram$ 
normaluser@e15e7e6bb951:~/exampleprogram$ # run govulncheck (does not work)
normaluser@e15e7e6bb951:~/exampleprogram$ govulncheck ./...
govulncheck is an experimental tool. Share feedback at https://go.dev/s/govulncheck-feedback.

Scanning for dependencies with known vulnerabilities...
govulncheck: vulncheck.Source: GetByModule("golang.org/x/text"): httpSource.GetByModule("golang.org/x/text"): Index(): mkdir /pkg: permission denied
normaluser@5374067b4b4f:~/exampleprogram$ echo "GOPATH?" $(go env GOPATH)
GOPATH? /home/normaluser/go
normaluser@5374067b4b4f:~/exampleprogram$ GOPATH=/home/normaluser/go govulncheck ./...
govulncheck is an experimental tool. Share feedback at https://go.dev/s/govulncheck-feedback.

Scanning for dependencies with known vulnerabilities...
No vulnerabilities found.

timothy-king commented 1 year ago

What are the contents of the GOENV file, which looks to be "/home/ej/.config/go/env" according to go env?

evanj commented 1 year ago

On my personal Linux desktop it contains:

CC=clang

In the clean Ubuntu container reproduction example the file $HOME/.config/go/env does not exist.

zpavlinovic commented 1 year ago

Thanks for the very detailed reproduction instructions! The fix should now be in. At the very least, I was able to follow reproduction steps without them resulting in an error/panic:

$ govulncheck ./...
govulncheck is an experimental tool. Share feedback at https://go.dev/s/govulncheck-feedback.

Scanning for dependencies with known vulnerabilities...
No vulnerabilities found.

evanj commented 1 year ago

Confirmed that this fixes my issue! Thanks for the rapid response!

golang / go