golang `stdlib` scan inconsistency for fresh go toolchain versions: bare metal vs docker

tommiv commented 7 months ago

I've recently faced a weird issue in our CI process – it reports some vulnerabilities for go v1.20, while the go.mod file claims go 1.21.4. At the same time, my local version on a dev machine reports "no vulnerabilities found".

So, given this /tmp/osv-issue/go.mod file (not much to see here, the original one is way bigger, but the issue persists even with a minimal one):

module some_module_name

go 1.21.4

require (
)

I have the following results:

[MacOS v14.1.1] [clean] bare metal binary, installed via homebrew:

osv-scanner -v
osv-scanner version: 1.4.3
commit: n/a
built at: n/a

Scanned /tmp/osv-issue/go.mod file and found 1 package
No vulnerabilities found

[MacOS v14.1.1] [clean] bare metal binary, manual download darwin_amd64 from github releases:

curl -OL https://github.com/google/osv-scanner/releases/download/v1.4.3/osv-scanner_1.4.3_darwin_amd64
chmod +x osv-scanner_1.4.3_darwin_amd64
./osv-scanner_1.4.3_darwin_amd64 -v
osv-scanner version: 1.4.3
commit: 6316373e47d7e3e4b4fd3630c4bbc10987738de6
built at: 2023-11-02T00:53:14Z

./osv-scanner_1.4.3_darwin_amd64 -L /tmp/osv-issue/go.mod
Scanned /tmp/osv-issue/go.mod file and found 1 package
No vulnerabilities found

[vulnerable] official docker way (copypasted some commands from docs)

docker run -it -v ${PWD}:/src ghcr.io/google/osv-scanner:v1.4.3 -v            
osv-scanner version: 1.4.3
commit: 6316373e47d7e3e4b4fd3630c4bbc10987738de6
built at: 2023-11-02T00:53:14Z

 docker run -it -v /tmp/osv-issue:/src ghcr.io/google/osv-scanner:v1.4.3 -L /src/go.mod
Scanned /src/go.mod file and found 1 package
╭──────────────────────────────┬──────┬───────────┬─────────┬─────────┬────────────╮
│ OSV URL                      │ CVSS │ ECOSYSTEM │ PACKAGE │ VERSION │ SOURCE     │
├──────────────────────────────┼──────┼───────────┼─────────┼─────────┼────────────┤
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.20.10 │ src/go.mod │
│                              │      │           │         │         │            │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.20.10 │ src/go.mod │
│                              │      │           │         │         │            │
╰──────────────────────────────┴──────┴───────────┴─────────┴─────────┴────────────╯

As you can see, the result in docker is quite different, while it runs binaries compiled against exactly the same codebase (based on the commit hash) – it reports vulnerabilities for go stdlib v.1.20.10.

It seems that osv-scanner uses the original golang toolchain to get the report (it fails to run if go is not installed), so I've tried to install golang in a clean container of alpine:latest and indeed it emits

go version
go version go1.20.11 linux/amd64

Vulnerabilities above are actually already fixed in v1.20.11 but the ghcr.io/google/osv-scanner:v1.4.3 image was built three weeks ago, probably before the patch had become available.

The bottom line here is: in my case our CI reports false positives.

P.S. TBH I have no idea how one could solve that with the official image, since it's on the alpine side to provide the latest go toolchain version, so I expect this issue will be closed as "won't fix", just wanted to bring the problem to maintainers attention.

another-rex commented 7 months ago

This seems to be a result of: https://github.com/google/osv-scanner/issues/453. The assumption is made that the go binary found on the PATH will be the version that is building the go package, which might not always be the case.

I think we want to do a couple of things here:

Remove golang from the docker image, as it is not necessary to even be there (example is the action.dockerfile image) (This will break/skip gouvlncheck call analysis though.)
Add an option to override the go version, so you can input your own actual go version to show correct results.

tommiv commented 7 months ago

@another-rex

Remove golang from the docker image, as it is not necessary to even be there

I don't know of osv-scanner internals, but this would definetely break golang stdlib checks mentioned in #453, here's the quick demo:

docker run -it alpine:latest
apk add curl
cd /tmp
curl -OL https://github.com/google/osv-scanner/releases/download/v1.4.3/osv-scanner_1.4.3_linux_amd64
chmod +x osv-scanner_1.4.3_linux_amd64
printf "module test\ngo 1.121\n" > go.mod
./osv-scanner_1.4.3_linux_amd64 -L go.mod

This will emit the following:

cannot get go standard library version, go might not be installed: exec: "go": executable file not found in $PATH
Scanned /tmp/go.mod file and found 0 packages
No package sources found, --help for usage information.

So, I guess, providing some way to switch to a desired go toolchain version is prefered. However I understood from #453 that as of now, there's no clear way to do this.

UPD: As an alternative, not suitable for everyone though, one can use golang:$VERSION-alpine image and just download osv-scanner from github releases. In this approach though it will be necessary to stick to a particular version of osv-scanner or use additional scripting/software to obtain the latest release tag from the github releases page, since github itself doesn't provide an alias url for the "latest" release.

another-rex commented 7 months ago

cannot get go standard library version, go might not be installed: exec: "go": executable file not found in $PATH

This error has been converted into a warning in the next release (https://github.com/google/osv-scanner/pull/622) (v1.5.0 is currently planned to release next week), and as long as there is a package in the go.mod file, it should successfully scan.

another-rex commented 7 months ago

After discussing this with the team, we are going to change the implementation to read the go.mod file for the go version, and if no patch version is specified in go.mod, we assume .0 patch version. This will replace finding the go version in $PATH proposed in #453.

It looks like after 1.21, go version in go.mod is enforced as the minimum version.

spencerschrock commented 4 months ago

I was playing around with this when trying out Go call analysis.

The toy project uses an old version of Go intentionally. Based on my understanding of https://osv.dev/vulnerability/GO-2023-2102: http.ListenAndServe is vulnerable for versions before 1.20.10 (or 1.21.3)

package main

import (
    "log"
    "net/http"
)

func main() {
    err := http.ListenAndServe(":8080", nil)
    if err != nil {
        log.Fatal(err)
    }
}

the example go.mod declares a vulnerable version:

module example.com/foo/bar

go 1.19

But osv-scanner seems to also care what my Go version is when determining reachability (go.mod still says 1.19).

$ go version
go version go1.22.0 linux/amd64

$ osv-scanner -r .
Scanning dir .
Scanned /tmp/go/go.mod file and found 1 package
Uncalled vulnerabilities 
│ https://osv.dev/GO-2023-2375 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2041 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2043 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2102 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2382 │      │ Go        │ stdlib  │ 1.19    │ go.mod │

Vs if I use an older Go toolchain (example go.mod is still 1.19).

$ go version
go version go1.21.2 linux/amd64
$ osv-scanner -r .
Scanning dir .
Scanned /tmp/go/go.mod file and found 1 package
(called vulnerabilities section)
│ https://osv.dev/GO-2023-2102 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2382 │      │ Go        │ stdlib  │ 1.19    │ go.mod │

Uncalled vulnerabilities
│ https://osv.dev/GO-2023-2375 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2041 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2043 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.19    │ go.mod │

Am I misunderstanding what #704 did? I'm trying to understand this behavior before enabling it for Scorecard because our weekly scan runs with the latest version of the Go toolchain. So I don't want to dismiss everything as uncalled based on the Go toolchain version we use.

hogo6002 commented 4 months ago

Thanks @spencerschrock for reporting this.

As expected, osv-scanner should not care what the local Go version is when determining reachability. It should only checks Go version that is defined in go.mod.

I think the issue here is because when we run govulncheck, we didn't set GOVERSION to the same one as go.mod (By default, GOVERSION represents the installed go version.). And govulncheck takes this variable to determine vulns.

To fix it, I will raise a PR to assign GOVERSION to the right value before executing govulncheck. How do you think @another-rex

$ export GOVERSION=go1.19.0
$ go run ./cmd/osv-scanner/ -r ../test3

│ https://osv.dev/GO-2023-2375 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2102 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2382 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ Uncalled vulnerabilities     │      │           │         │         │                 │
│ https://osv.dev/GO-2023-2041 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2043 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │

exit status 1

$ export GOVERSION=go1.22.0
$ go run ./cmd/osv-scanner/ -r ../test3
│ Uncalled vulnerabilities     │      │           │         │         │                 │
│ https://osv.dev/GO-2023-2375 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2041 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2043 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2102 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2382 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │

spencerschrock commented 4 months ago

Ah I didn't know about GOVERSION. I tried clearing the Env variables mentioned in the docs, thinking it had to do with GOROOT (or similar) but setting a new value for GOVERSION sounds perfect. https://pkg.go.dev/golang.org/x/vuln/scan#Cmd

google / osv-scanner