search

open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.02k stars 2.33k forks source link

make test fails with duplicate symbol #5425

Closed codeboten closed 2 years ago

codeboten commented 3 years ago

Describe the bug Running make test fails on MacOS arm64 with a duplicate symbol error.

make test
go test -race -timeout 60s --tags=containers_image_openpgp,exclude_graphdriver_btrfs,exclude_graphdriver_devicemapper ./...
?       github.com/open-telemetry/opentelemetry-collector-contrib/cmd/otelcontribcol    [no test files]
# github.com/open-telemetry/opentelemetry-collector-contrib/internal/components.test
/opt/homebrew/Cellar/go/1.17.1/libexec/pkg/tool/darwin_arm64/link: running clang failed: exit status 1
duplicate symbol '_readdrivestat' in:
    /var/folders/f2/5z5mvgcx6lx24rc_2_xzv2g40000gp/T/go-link-1335735787/000029.o
    /var/folders/f2/5z5mvgcx6lx24rc_2_xzv2g40000gp/T/go-link-1335735787/000032.o
ld: 1 duplicate symbol for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Steps to reproduce

  1. Clone the repo on a MacOS arm64 machine
  2. Run make test

What did you expect to see? The tests passing.

What did you see instead? The error pasted above.

What version did you use? This has been happening for some time, but I tested it last w/ main at version 1f91c2701d812bc7e7ab0ffa387d6999c96cef3b

Environment OS: MacOS arm64 Compiler(if manually compiled): go 1.17.1

mx-psi commented 3 years ago

Reported on open-telemetry/opentelemetry-collector/discussions/4018 (also happens on amd64). I think it makes more sense to keep this issue and close the other discussion since it seems to be a contrib-only problem.

mx-psi commented 3 years ago

After some git bisect it seems like it starts happening after #4386 (I am guessing the culprit is some dependency, maybe golang.org/x/sys?)

codeboten commented 3 years ago

Can confirm this appears to be MacOS specific. I just ran the tests inside a Docker container on my mac and it appears to be fine:

root@14b3bb7009e0:/app# make test
go test -race -timeout 300s --tags=containers_image_openpgp,exclude_graphdriver_btrfs,exclude_graphdriver_devicemapper ./...
?       github.com/open-telemetry/opentelemetry-collector-contrib/cmd/otelcontribcol    [no test files]
ok      github.com/open-telemetry/opentelemetry-collector-contrib/internal/components   0.251s
ok      github.com/open-telemetry/opentelemetry-collector-contrib/internal/version  0.020s [no tests to run]
Aneurysm9 commented 3 years ago

It seems to be related to the https://github.com/shirou/gopsutil library used by the host metrics receiver. It can be avoided by setting CGOENABLED=0, though I'm not sure that's the correct resolution.

mx-psi commented 3 years ago

I believe I found the root cause and opened a PR upstream to fix this (shirou/gopsutil#1137). It would be nice to get a second pair of eyes to test and review the patch (cc @codeboten)

codeboten commented 3 years ago

@mx-psi nice! I will give it a try later today.

anuraaga commented 3 years ago

I seem to still get this issue, or something similar. Does anyone know if this is expected to already be fixed?

# github.com/open-telemetry/opentelemetry-collector-contrib/internal/components.test
/usr/local/Cellar/go/1.17/libexec/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
duplicate symbol '_get_temperature' in:
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000024.o
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000031.o
duplicate symbol '_open_smc' in:
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000024.o
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000031.o
duplicate symbol '_close_smc' in:
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000024.o
    /var/folders/sd/93mdclc148x9rdghp0zcl1l55n4fjn/T/go-link-3937615906/000031.o
ld: 3 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
mx-psi commented 3 years ago

Unfortunately, I think #5570 reintroduced this issue (in a different way) because of these symbols. I was hoping the bump to gopsutil/v3 would fix this but it seems somewhere in our dependency tree we still use gopsutil v2.

The fix upstream should be pretty similar to the one for the previous duplicate symbol but I wonder if we want to set CGO_ENABLED=0 for make test so that this doesn't keep coming up.

anuraaga commented 3 years ago

Unfortunately disabling cgo for make test also is awkward ><

➜  opentelemetry-collector-contrib git:(wireawsxray) CGO_ENABLED=0 make test
go test -race -timeout 300s --tags="" ./...
go test: -race requires cgo; enable cgo by setting CGO_ENABLED=1
make: *** [test] Error 2
➜  opentelemetry-collector-contrib git:(
mx-psi commented 2 years ago

@anuraaga shirou/gopsutil#1176 may solve the issues on the symbols mentioned on your last message. Would you mind checking if the issue still happens to you on main, and, if so, if the issue is fixed by bumping gopsutil to v3.21.11?

anuraaga commented 2 years ago

@mx-psi Thanks for the note, make test seems to work fine for me now on a Mac! Though lately I've switched to using remote development against a Linux VM so may not find future regressions :P