Test failures in Go 1.8

[nathany]

After upgrading our QA environment to Go 1.8 (rc2), we lost insight into New Relic metrics.

Running tests on master (commit 7d12ae2201fc160e486197614a6f65afcf3f8170) with Go 1.7 is successful (❯ go17 test ./...).

However, tests are failing when running with Go 1.8 (rc2).

❯ go version     
go version go1.8rc2 darwin/amd64

❯ go test ./...  
--- FAIL: TestCopyConfigReferenceFieldsPresent (0.00s)
    internal_config_test.go:119: [{"pid":123,"language":"go","agent_version":"0.2.2","host":"my-hostname","settings":{"AppName":"my appname","Attributes":{"Enabled":true,"Exclude":["2"],"Include":["1"]},"CustomInsightsEvents":{"Enabled":true},"DatastoreTracer":{"DatabaseNameReporting":{"Enabled":true},"InstanceReporting":{"Enabled":true},"QueryParameters":{"Enabled":true},"SlowQuery":{"Enabled":true,"Threshold":10000000}},"Enabled":true,"ErrorCollector":{"Attributes":{"Enabled":true,"Exclude":["6"],"Include":["5"]},"CaptureEvents":true,"Enabled":true,"IgnoreStatusCodes":[404,405]},"HighSecurity":false,"HostDisplayName":"","Labels":{"zip":"zap"},"Logger":"*logger.logFile","RuntimeSampler":{"Enabled":true},"TransactionEvents":{"Attributes":{"Enabled":true,"Exclude":["4"],"Include":["3"]},"Enabled":true},"TransactionTracer":{"Attributes":{"Enabled":true,"Exclude":["8"],"Include":["7"]},"Enabled":true,"SegmentThreshold":2000000,"StackTraceThreshold":500000000,"Threshold":{"Duration":500000000,"IsApdexFailing":true}},"Transport":"*http.Transport","UseTLS":true,"Utilization":{"BillingHostname":"","DetectAWS":true,"DetectDocker":true,"LogicalProcessors":0,"TotalRAMMIB":0}},"app_name":["my appname"],"high_security":false,"labels":[{"label_type":"zip","label_value":"zap"}],"environment":[["runtime.Compiler","comp"],["runtime.GOARCH","arch"],["runtime.GOOS","goos"],["runtime.Version","vers"],["runtime.NumCPU",8]],"identifier":"my appname","utilization":{"metadata_version":2,"logical_processors":16,"total_ram_mib":1024,"hostname":"my-hostname"}}]
--- FAIL: TestCopyConfigReferenceFieldsAbsent (0.00s)
    internal_config_test.go:208: [{"pid":123,"language":"go","agent_version":"0.2.2","host":"my-hostname","settings":{"AppName":"my appname","Attributes":{"Enabled":true,"Exclude":null,"Include":null},"CustomInsightsEvents":{"Enabled":true},"DatastoreTracer":{"DatabaseNameReporting":{"Enabled":true},"InstanceReporting":{"Enabled":true},"QueryParameters":{"Enabled":true},"SlowQuery":{"Enabled":true,"Threshold":10000000}},"Enabled":true,"ErrorCollector":{"Attributes":{"Enabled":true,"Exclude":null,"Include":null},"CaptureEvents":true,"Enabled":true,"IgnoreStatusCodes":null},"HighSecurity":false,"HostDisplayName":"","Labels":null,"Logger":null,"RuntimeSampler":{"Enabled":true},"TransactionEvents":{"Attributes":{"Enabled":true,"Exclude":null,"Include":null},"Enabled":true},"TransactionTracer":{"Attributes":{"Enabled":true,"Exclude":null,"Include":null},"Enabled":true,"SegmentThreshold":2000000,"StackTraceThreshold":500000000,"Threshold":{"Duration":500000000,"IsApdexFailing":true}},"Transport":null,"UseTLS":true,"Utilization":{"BillingHostname":"","DetectAWS":true,"DetectDocker":true,"LogicalProcessors":0,"TotalRAMMIB":0}},"app_name":["my appname"],"high_security":false,"environment":[["runtime.Compiler","comp"],["runtime.GOARCH","arch"],["runtime.GOOS","goos"],["runtime.Version","vers"],["runtime.NumCPU",8]],"identifier":"my appname","utilization":{"metadata_version":2,"logical_processors":16,"total_ram_mib":1024,"hostname":"my-hostname"}}]
FAIL
FAIL    github.com/newrelic/go-agent    0.037s
?       github.com/newrelic/go-agent/examples/server    [no test files]
?       github.com/newrelic/go-agent/examples/short-lived-process   [no test files]
ok      github.com/newrelic/go-agent/internal   0.042s
?       github.com/newrelic/go-agent/internal/crossagent    [no test files]
ok      github.com/newrelic/go-agent/internal/jsonx 0.012s
?       github.com/newrelic/go-agent/internal/logger    [no test files]
ok      github.com/newrelic/go-agent/internal/sysinfo   0.030s
?       github.com/newrelic/go-agent/internal/tools/rules   [no test files]
?       github.com/newrelic/go-agent/internal/tools/utilization [no test files]
ok      github.com/newrelic/go-agent/internal/utilization   0.319s

Go 1.8 (rc2) announcement: https://groups.google.com/forum/#!topic/golang-announce/iI13Nx0BP2E

[nathany]

Aside: Is there any sort of CI system setup to run these tests? Would you consider a test matrix for multiple Go versions using something like TravisCI? Let me know if you would like a PR for that.

[nathany]

The failure in TestCopyConfigReferenceFieldsPresent is due to the way numbers are being encoded in Go 1.8.

"Marshal encodes floating-point numbers using the same format as in ES6, preferring decimal (not exponential) notation for a wider range of values. In particular, all floating-point integers up to 264 format the same as the equivalent int64 representation." https://beta.golang.org/doc/go1.8

I don't imagine that related to why we stopped seeing any metrics in New Relic. It's just a change that the string comparison in this test doesn't account for.

[willnewrelic]

Hi @nathany

Thanks for the heads up! We will fix this shortly.

We have a CI system internally: We are looking into what system would work for both public and private development.

[nathany]

Thanks Will.

I think the issue we experienced with losing New Relic data wasn't related to Go 1.8, but instead from switching from golang:1.7.4-alpine (Docker) to alpine:3.5. I don't see why New Relic go-agent would require the Go compiler, but perhaps there is something else it needs that exists in the golang image?

As far as this issue, maybe using smaller (single digit?) values would satisfy the tests across both Go 1.7 and Go 1.8, assuming that the tests stick with hard coded JSON output.

[willnewrelic]

@nathany I don't have any insight into how the Docker change could affect agent behavior. Are you still experiencing this problem? If so, let's address it in a separate issue or support ticket.

Go 1.8 tests fixed as part of the latest 1.6.0 release! Thanks for bringing this to our attention!

[nathany]

Thanks @willnewrelic.