Issue: [detector: gcp] "cannot merge resource due to conflicting Schema URL" in 1.18.0

[utsushiiro]

Description

After upgrading go.opentelemetry.io/contrib/detectors/gcp to v1.18.0, an error occurred in the application running on Cloud Run: "cannot merge resource due to conflicting Schema URL".

go.opentelemetry.io/contrib/detectors/gcp@v1.18.0 uses go.opentelemetry.io/otel/semconv/v1.17.0, while its dependency go.opentelemetry.io/otel/sdk@v1.17.0 uses go.opentelemetry.io/otel/semconv/v1.21.0. Is this version mismatch the cause of the error?

Related PRs:

• https://github.com/open-telemetry/opentelemetry-go/pull/4408

Environment

My Application is running on Cloud Run. Docker image is based on gcr.io/distroless/static-debian11.

• Go Version: 1.21
• opentelemetry.io/contrib/detectors/gcp version: 1.18.0
• go.opentelemetry.io/otel/sdk version: 1.17.0

Code

    res, err := resource.New(
        ctx,
        resource.WithDetectors(gcp.NewDetector()), // <- semconv/v1.17.0 is used 
        resource.WithTelemetrySDK(), // <- semconv/v1.21.0 is used
        resource.WithAttributes(
            semconv.ServiceNameKey.String(serviceName),
            attribute.String("environment", env),
        ),
    )
    ...

[MrAlias]

This is a known issue: https://github.com/open-telemetry/opentelemetry-go/issues/2341

https://github.com/MrAlias/otel-schema-utils was created to help while the specification and this project come to define its own translation approach.

[maximkosov]

We also hit this issue yesterday.

@MrAlias how can we use https://github.com/MrAlias/otel-schema-utils to resolve this issue?

Also, fyi, it means that gcp detector 1.18 is broken because it requires sdk 1.17 in go.mod and apparently it is not compatible with that version of sdk.

[MrAlias]

cc @dashpole

[dashpole]

This will be fixed by https://github.com/open-telemetry/opentelemetry-go-contrib/pull/4263

[MrAlias]

@MrAlias how can we use https://github.com/MrAlias/otel-schema-utils to resolve this issue?

module main

go 1.21.0

require (
    github.com/MrAlias/otel-schema-utils v0.2.1-alpha
    go.opentelemetry.io/contrib/detectors/gcp v1.18.0
    go.opentelemetry.io/otel v1.17.0
    go.opentelemetry.io/otel/sdk v1.17.0
)

require (
    cloud.google.com/go/compute v1.14.0 // indirect
    cloud.google.com/go/compute/metadata v0.2.3 // indirect
    github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.18.0 // indirect
    github.com/Masterminds/semver/v3 v3.2.1 // indirect
    github.com/go-logr/logr v1.2.4 // indirect
    github.com/go-logr/stdr v1.2.2 // indirect
    go.opentelemetry.io/otel/metric v1.17.0 // indirect
    go.opentelemetry.io/otel/schema v0.0.5 // indirect
    go.opentelemetry.io/otel/trace v1.17.0 // indirect
    golang.org/x/sys v0.11.0 // indirect
    gopkg.in/yaml.v2 v2.4.0 // indirect
)

package main

import (
    "context"
    "log"

    "github.com/MrAlias/otel-schema-utils/schema"
    "go.opentelemetry.io/contrib/detectors/gcp"
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/sdk/resource"
    semconv "go.opentelemetry.io/otel/semconv/v1.17.0"
)

const (
    serviceName = "my-service"
    env         = "prod"
)

func must(res *resource.Resource, err error) *resource.Resource {
    if err != nil {
        log.Fatal(err)
    }
    return res
}

func main() {
    conv := schema.NewConverter(schema.NewLocalClient())

    ctx := context.Background()
    res := must(conv.MergeResources(
        ctx, semconv.SchemaURL,
        must(gcp.NewDetector().Detect(ctx)),
        must(resource.New(
            ctx,
            resource.WithTelemetrySDK(),
            resource.WithAttributes(
                semconv.ServiceNameKey.String(serviceName),
                attribute.String("environment", env),
            ),
        ),
        ),
    ))
    log.Println(res.SchemaURL())
}

$ go run .
2023/08/31 13:48:23 https://opentelemetry.io/schemas/1.17.0

[dustinmoris]

Is there any other way of getting this to work right now without having to import yet another third party dependency like https://github.com/MrAlias/otel-schema-utils?

Any workarounds, etc.?

[maximkosov]

@dustinmoris I just removed resource.WithTelemetrySDK(). It is only adding version and language of open telemetry library, which is always the same in our case.

[dustinmoris]

@maximkosov Thanks, that's probably better than what I did just now. I copied the entire source code from detectors/gcp into my own project, changed the semconv to 1.21.0 and renamed FaaSIDKey to FaaSInstanceKey in 4 places as per a comment in another issue and that worked for me too.

[anuraaga]

@dustinmoris Also can consider manually running contrib detectors and only using their attributes, it allows all the data to still be present without copying code

    gcpRes, err := gcp.NewDetector().Detect(ctx)
    if err != nil {
        panic(err)
    }

    res, err := resource.New(ctx,
        resource.WithAttributes(gcpRes.Attributes()...),
        resource.WithTelemetrySDK(),
        resource.WithHost(),
        resource.WithContainer(),
        resource.WithProcess(),
        resource.WithOS(),
        resource.WithFromEnv(),
    )

It's too bad the original issue https://github.com/open-telemetry/opentelemetry-go/issues/2341 doesn't have more traction as it does seem it's creating a lot of unhappy users.