open-telemetry / opentelemetry-go

OpenTelemetry Go API and SDK
https://opentelemetry.io/
Apache License 2.0
5.14k stars 1.04k forks source link

instruments creation may cause panic when setting meter provider #5757

Closed XSAM closed 1 week ago

XSAM commented 2 weeks ago

Description

This bug was found in CI after running TestMeterConcurrentSafe test and was introduced in #5754, where creating instruments while calling setDelegate method.

This is because creating instruments can be locked in this line:

https://github.com/open-telemetry/opentelemetry-go/blob/e47618fc36af51d17ecdcc7299bbf706397e1cb1/internal/global/meter.go#L350

While setDelegate is cleaning instruments map:

https://github.com/open-telemetry/opentelemetry-go/blob/e47618fc36af51d17ecdcc7299bbf706397e1cb1/internal/global/meter.go#L144

Then, the remaining code in the instrument creation method would trigger panic after the mutex is unlocked:

https://github.com/open-telemetry/opentelemetry-go/blob/e47618fc36af51d17ecdcc7299bbf706397e1cb1/internal/global/meter.go#L360

Steps To Reproduce

  1. Run
go test -timeout 30s -run ^TestMeterConcurrentSafe$ go.opentelemetry.io/otel/internal/global -count 1000

Expected behavior

No Panic

Proposal

XSAM commented 2 weeks ago

This blocks the release.