Compiling Grafana using FIPS with Microsoft Go 1.21

[devinacosta]

I am a Linux Engineer that builds certain Go Packages at work for our FedRAMP program. I am trying to get some clarification around an issue I am seeing when building Grafana using RPM SPEC with Microsoft GO 1.21.10.

My previous build of Grafana used Microsoft Go Lang version 1.20.5, and I was able to successfully build the RPM using the following:

export IMPORTPATH=%{_builddir}/grafana-%{version}
export BUILDFLAGS="-v -p 4 -x -buildmode=pie -mod=vendor"
export GOPATH=%{_builddir}/go:%{_builddir}/contrib
export GOBIN=/usr/local/go/bin
export GOEXPERIMENT=opensslcrypto
export GOFIPS=1
wire gen -tags 'oss' ./pkg/server ./pkg/cmd/grafana-cli/runner

see grafana-X.Y.Z/pkg/build/cmd.go
export LDFLAGS="-X main.version=%{version} -X main.buildstamp=${SOURCE_DATE_EPOCH}"
for cmd in grafana grafana-cli grafana-server; do
    %gobuild -o %{_builddir}/bin/${cmd} ./pkg/cmd/${cmd}
done

However as Grafana 10.4.5 now uses Go Lang 1.21.10, those build settings above do not work for me any longer. The information that I gathered from reading https://github.com/microsoft/go/blob/microsoft/main/eng/doc/fips/README.md, it seems like I should only need to set:

GOEXPERIMENT=systemcrypto

in order to get a successful FIPS complaint build? It appears if i set GOFIPS=1 during the build process it will fail horribly.

+ cd /root/rpmbuild/BUILD/grafana-10.2.3
+ export IMPORTPATH=/root/rpmbuild/BUILD/grafana-10.2.3
+ IMPORTPATH=/root/rpmbuild/BUILD/grafana-10.2.3
+ export 'BUILDFLAGS=-v -p 4 -x -buildmode=pie -mod=vendor'
+ BUILDFLAGS='-v -p 4 -x -buildmode=pie -mod=vendor'
+ export GOPATH=/root/rpmbuild/BUILD/go:/root/rpmbuild/BUILD/contrib
+ GOPATH=/root/rpmbuild/BUILD/go:/root/rpmbuild/BUILD/contrib
+ export GOBIN=/usr/local/go/bin
+ GOBIN=/usr/local/go/bin
+ export GOEXPERIMENT=systemcrypto
+ GOEXPERIMENT=systemcrypto
+ export GOOS=linux
+ GOOS=linux
+ export GOFIPS=1
+ GOFIPS=1
+ wire gen -tags oss ./pkg/server ./pkg/cmd/grafana-cli/runner
wire: err: exit status 2: stderr: panic: FIPS mode requested (environment variable GOFIPS=1) but no supported crypto backend is enabled

        goroutine 1 [running]:
        crypto/internal/backend.init.0()
                crypto/internal/backend/common.go:21 +0x65

wire: generate failed
error: Bad exit status from /var/tmp/rpm-tmp.eIFDsO (%build)

RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.eIFDsO (%build)

I am just trying to understand what options I should ONLY be setting during the build process to ensure I am getting as complaint of a build as i can? Does GOFIPS=1 need to be set during the build, or is that only needed during the running of the application? Any advise would be appreciated.

[dagood]

To recap a few things I wrote in the golang-nuts thread:

The issue doesn't seem related to Grafana, but rather because wire was built with the Microsoft fork of Go but without specifying a backend [and now GOFIPS=1 is set]. wire isn't able to be compatible with FIPS without a backend, but it sees that FIPS is requested, so it fails safe. It isn't clear what the caller's intent is and failing is an opportunity to catch a mistake. You should either:

1. not set GOFIPS=1 until after calling wire (if at all!) or
2. build wire with GOEXPERIMENT=systemcrypto.

I would default to (1). But if you are trying to make a FIPS compliant package build process, (2) would be the step towards that.

Whether or not you need GOFIPS=1 at all depends on the purpose of your script/build process.

using GOFIPS=1 worked just fine on Go 1.20.5, however appears not to be the case anymore.

Yes, we only added this failsafe as of 1.21 of Microsoft Go. The first bullet in the 1.21 changelog has some details.

To lay out the behavior of wire a bit more explicitly:

• wire is a Go program, and in this case Microsoft Go was used to build it.
• With Microsoft Go 1.20.5 and GOFIPS=1, wire ignores the flag.
• With Microsoft Go 1.21.10 and GOFIPS=1, wire sees the flag, knows that it doesn't support FIPS mode (because it was built without GOEXPERIMENT=systemcrypto), and alerts you to the impossible FIPS mode request by failing.
• With Microsoft Go 1.21.10 and no GOFIPS, wire will run with the Go standard library crypto functions.

---

Thanks for filing over here. 😄 I'm going to do my best to answer in more detail. Apologies in advance--some of this might come across as pedantic, but FIPS can be fiddly and sensitive.

it seems like I should only need to set: GOEXPERIMENT=systemcrypto in order to get a successful FIPS complaint build?

We can't really say whether your app will be FIPS compliant. That goes beyond what we currently think can be done by tools alone. Some discussion about that here: https://github.com/microsoft/go/issues/428, and some more info about the "compliant" and "certified" language in the disclaimer.

Generally, the minimum conditions for building a FIPS compliant app with Microsoft Go 1.21 are:

• Use GOEXPERIMENT=systemcrypto.
• Make sure the system it runs on will enable FIPS mode in an expected way.

  • Linux FIPS mode sets the content of /proc/sys/crypto/fips_enabled to 1. The Go runtime reads this file. (doc)

• Either don't use TLS (or use anything that uses TLS) (unlikely), or enable fipsonly TLS.
  • (If you are using 1.22+, FIPS-only TLS is enabled automatically.)
• Review the app to ensure its usage of crypto is FIPS compliant.
  • We have created a FIPS User Guide to try to help with this.

On Linux, this results in an app that will always use OpenSSL to implement standard library crypto functions and will enable FIPS mode if the Linux system is configured to run in FIPS mode.

(There are alternatives to most of these bullet points--it's just an example of a minimal process.)

Does GOFIPS=1 need to be set during the build, or is that only needed during the running of the application?

I think it's safe to assume that you don't need the build process itself to be FIPS compliant, only the resulting binaries. (If you're delivering a distro builder (something like Yocto?) then maybe you do need the build process itself to be FIPS compliant, but this would be unusual.)

GOFIPS=1 never needs to be used during the process of building a FIPS compliant app. All it does is force the initialization process of an executing Go program that was built by Microsoft Go to try to enable FIPS mode and fail if that can't be done. Even if this is necessary (e.g. the distro doesn't support /proc/sys/crypto/fips_enabled, or you want to test out FIPS mode on a machine that isn't FIPS enabled) then you would set GOFIPS=1 at runtime, not during the build.