[CI] Failures in build-and-test / unittest-matrix don't cause the job to fail

[dmitryax]

Look at the failing TestLoadInvalidConfig_NoScrapers test in https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/3700493629/jobs/6269016994 :

=== RUN   TestLoadInvalidConfig_NoScrapers
    config_test.go:112: 
            Error Trace:    /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/config_test.go:112
            Error:          "receivers::hostmetrics: must specify at least one scraper when using hostmetrics receiver" does not contain "receiver \"hostmetrics\" has invalid configuration: must specify at least one scraper when using hostmetrics receiver"
            Test:           TestLoadInvalidConfig_NoScrapers
--- FAIL: TestLoadInvalidConfig_NoScrapers (0.00s)

[github-actions[bot]]

Pinging code owners for exporter/loadbalancing: @jpkrohling. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dmitryax]

Looks like we released 0.67.0 with the following failing unit tests:

=== RUN   TestLoadInvalidConfig_NoScrapers
    config_test.go:112: 
            Error Trace:    /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/config_test.go:112
            Error:          "receivers::hostmetrics: must specify at least one scraper when using hostmetrics receiver" does not contain "receiver \"hostmetrics\" has invalid configuration: must specify at least one scraper when using hostmetrics receiver"
            Test:           TestLoadInvalidConfig_NoScrapers
--- FAIL: TestLoadInvalidConfig_NoScrapers (0.00s)

=== RUN   TestConsumeLogsUnexpectedExporterType
    log_exporter_test.go:198: 
            Error Trace:    /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/loadbalancingexporter/log_exporter_test.go:198
            Error:          Error message not equal:
                            expected: "unable to export logs, unexpected exporter type: expected *exporter.Logs but got loadbalancingexporter.mockComponent"
                            actual  : "unable to export logs, unexpected exporter type: expected *component.LogsExporter but got loadbalancingexporter.mockComponent"
            Test:           TestConsumeLogsUnexpectedExporterType
--- FAIL: TestConsumeLogsUnexpectedExporterType (0.00s)

=== RUN   TestConsumeTracesUnexpectedExporterType
    trace_exporter_test.go:300: 
            Error Trace:    /home/runner/work/opentelemetry-collector-contrib/opentelemetry-collector-contrib/exporter/loadbalancingexporter/trace_exporter_test.go:300
            Error:          Error message not equal:
                            expected: "expected *exporter.Traces but got loadbalancingexporter.mockComponent"
                            actual  : "expected *component.TracesExporter but got loadbalancingexporter.mockComponent"
            Test:           TestConsumeTracesUnexpectedExporterType
--- FAIL: TestConsumeTracesUnexpectedExporterType (0.00s)

See https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/3659745132/jobs/6186112298

Failing hostmetrics test is not a problem. But I need help from the routing processor owner to assess the impact

[dmitryax]

My first guess is that the issue is caused by https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/16731

[MovieStoreGuy]

Ooooh, I think I have an idea what has happened here.

Due to the fact that:

go test -race -timeout 300s --tags="" -v ./... 2>&1 | tee -a ./foresight-test-report.txt; \

go test pipes the result to tee, make doesn't capture the exit failure from go test but from tee

[dmitryax]

I think the tests broken in 0.67.0 are safe to not worry about 0.67.0 release

[dmitryax]

We are done. CI is back :)