search

artefactual-sdps / enduro

A tool to support ingest and automation in digital preservation workflows
https://enduro.readthedocs.io/
Apache License 2.0
4 stars 3 forks source link

Upgrade artefactual-sdps/temporal-activities #1011

Closed jraddaoui closed 2 months ago

jraddaoui commented 2 months ago

Upgrade github.com/artefactual-sdps/temporal-activities after the re-org, which also upgrades google.golang.org/grpc and other deps. Ignore grpc.DialContext and grpc.WithBlock deprecated warning.

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 41.17647% with 20 lines in your changes missing coverage. Please review.

Project coverage is 53.06%. Comparing base (e829a58) to head (9b674ba).

Files Patch % Lines
cmd/enduro-am-worker/main.go 0.00% 10 Missing :warning:
cmd/enduro-a3m-worker/main.go 0.00% 6 Missing :warning:
internal/telemetry/traces.go 0.00% 2 Missing :warning:
internal/a3m/client.go 0.00% 1 Missing :warning:
internal/workflow/processing.go 93.33% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1011 +/- ## ========================================== - Coverage 53.19% 53.06% -0.14% ========================================== Files 102 101 -1 Lines 5856 5814 -42 ========================================== - Hits 3115 3085 -30 + Misses 2483 2476 -7 + Partials 258 253 -5 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

jraddaoui commented 2 months ago

Upgrading github.com/artefactual-sdps/temporal-activities, several dependencies also got updated:

$ go get github.com/artefactual-sdps/temporal-activities
go: upgraded github.com/artefactual-labs/bagit-gython v0.0.0-20240723112522-d8d3815a59e6 => v0.2.0
go: upgraded github.com/artefactual-sdps/temporal-activities v0.0.0-20240722184934-3e9ad5a6dcda => v0.0.0-20240821162351-47302711bc7b
go: upgraded github.com/aws/aws-sdk-go v1.50.36 => v1.55.5
go: upgraded github.com/aws/aws-sdk-go-v2 v1.26.1 => v1.30.3
go: upgraded github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.2 => v1.6.3
go: upgraded github.com/aws/aws-sdk-go-v2/config v1.27.11 => v1.27.27
go: upgraded github.com/aws/aws-sdk-go-v2/credentials v1.17.11 => v1.17.27
go: upgraded github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.1 => v1.16.11
go: upgraded github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.16.9 => v1.17.10
go: upgraded github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.5 => v1.3.15
go: upgraded github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.5 => v2.6.15
go: upgraded github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.5 => v1.3.15
go: upgraded github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.2 => v1.11.3
go: upgraded github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.3.7 => v1.3.17
go: upgraded github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.7 => v1.11.17
go: upgraded github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.17.5 => v1.17.15
go: upgraded github.com/aws/aws-sdk-go-v2/service/s3 v1.53.1 => v1.58.3
go: upgraded github.com/aws/aws-sdk-go-v2/service/sso v1.20.5 => v1.22.4
go: upgraded github.com/aws/aws-sdk-go-v2/service/ssooidc v1.23.4 => v1.26.4
go: upgraded github.com/aws/aws-sdk-go-v2/service/sts v1.28.6 => v1.30.3
go: upgraded github.com/aws/smithy-go v1.20.2 => v1.20.3
go: upgraded github.com/cespare/xxhash/v2 v2.2.0 => v2.3.0
go: upgraded github.com/go-logr/logr v1.4.1 => v1.4.2
go: upgraded github.com/go-sql-driver/mysql v1.8.0 => v1.8.1
go: upgraded github.com/googleapis/gax-go/v2 v2.12.2 => v2.13.0
go: upgraded go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 => v0.53.0
go: upgraded go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 => v0.53.0
go: upgraded go.opentelemetry.io/otel v1.24.0 => v1.28.0
go: upgraded go.opentelemetry.io/otel/metric v1.24.0 => v1.28.0
go: upgraded go.opentelemetry.io/otel/trace v1.24.0 => v1.28.0
go: upgraded gocloud.dev v0.37.0 => v0.39.0
go: upgraded golang.org/x/crypto v0.22.0 => v0.26.0
go: upgraded golang.org/x/mod v0.16.0 => v0.17.0
go: upgraded golang.org/x/net v0.24.0 => v0.28.0
go: upgraded golang.org/x/oauth2 v0.18.0 => v0.22.0
go: upgraded golang.org/x/sync v0.7.0 => v0.8.0
go: upgraded golang.org/x/sys v0.22.0 => v0.24.0
go: upgraded golang.org/x/text v0.14.0 => v0.17.0
go: upgraded golang.org/x/time v0.5.0 => v0.6.0
go: upgraded golang.org/x/tools v0.19.0 => v0.21.1-0.20240508182429-e35e4ccd0d2d
go: upgraded golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 => v0.0.0-20240716161551-93cc26a95ae9
go: upgraded google.golang.org/api v0.169.0 => v0.191.0
go: upgraded google.golang.org/genproto/googleapis/api v0.0.0-20240401170217-c3f982113cda => v0.0.0-20240812133136-8ffd90a71988
go: upgraded google.golang.org/genproto/googleapis/rpc v0.0.0-20240401170217-c3f982113cda => v0.0.0-20240812133136-8ffd90a71988
go: upgraded google.golang.org/grpc v1.63.2 => v1.65.0
go: upgraded google.golang.org/protobuf v1.33.0 => v1.34.2

That broke the build process:

#16 [build-enduro 1/1] RUN --mount=type=cache,target=/go/pkg/mod    --mount=type=cache,target=/root/.cache/go-build     go build    -trimpath   -ldflags="-X 'github.com/artefactual-sdps/enduro/internal/version.Long=0.1.0-te829a5822' -X 'github.com/artefactual-sdps/enduro/internal/version.Short=0.1.0' -X 'github.com/artefactual-sdps/enduro/internal/version.GitCommit=e829a5822546e514a749a09717aef11655fc2051-dirty'"    -o /out/enduro  ./cmd/enduro
#16 sha256:00c06629b76450c9543845480c891e52ce505432519655093c1e06c65e14e8f8
#16 1.009 # go.opentelemetry.io/otel/sdk/trace
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/sdk@v1.24.0/trace/span.go:165:20: cannot use (*recordingSpan)(nil) (value of type *recordingSpan) as ReadWriteSpan value in variable declaration: *recordingSpan does not implement ReadWriteSpan (missing method AddLink)
#16 1.009       have addLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009       want AddLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/sdk@v1.24.0/trace/span.go:804:20: cannot use nonRecordingSpan{} (value of type nonRecordingSpan) as "go.opentelemetry.io/otel/trace".Span value in variable declaration: nonRecordingSpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/sdk@v1.24.0/trace/tracer.go:50:21: impossible type assertion: p.(*recordingSpan)
#16 1.009   *recordingSpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)
#16 1.009       have addLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009       want AddLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/sdk@v1.24.0/trace/tracer.go:120:10: cannot use tr.newNonRecordingSpan(sc) (value of type nonRecordingSpan) as "go.opentelemetry.io/otel/trace".Span value in return statement: nonRecordingSpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)
#16 1.009 /go/pkg/mod/go.opentelemetry.io/otel/sdk@v1.24.0/trace/tracer.go:122:9: cannot use tr.newRecordingSpan(psc, sc, name, samplingResult, config) (value of type *recordingSpan) as "go.opentelemetry.io/otel/trace".Span value in return statement: *recordingSpan does not implement "go.opentelemetry.io/otel/trace".Span (missing method AddLink)
#16 1.009       have addLink("go.opentelemetry.io/otel/trace".Link)
#16 1.009       want AddLink("go.opentelemetry.io/otel/trace".Link)
#16 ERROR: executor failed running [/bin/sh -c go build     -trimpath   -ldflags="-X '${VERSION_PATH}.Long=${VERSION_LONG}' -X '${VERSION_PATH}.Short=${VERSION_SHORT}' -X '${VERSION_PATH}.GitCommit=${VERSION_GIT_HASH}'"     -o /out/enduro  ./cmd/enduro]: exit code: 1

Which I could fix upgrading all the other go.opentelemetry.io dependencies:

$ go get go.opentelemetry.io/otel/sdk
go: upgraded go.opentelemetry.io/otel v1.28.0 => v1.29.0
go: upgraded go.opentelemetry.io/otel/metric v1.28.0 => v1.29.0
go: upgraded go.opentelemetry.io/otel/sdk v1.24.0 => v1.29.0
go: upgraded go.opentelemetry.io/otel/trace v1.28.0 => v1.29.0

$ go get go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
go: upgraded github.com/cenkalti/backoff/v4 v4.2.1 => v4.3.0
go: upgraded github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1 => v2.22.0
go: upgraded go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.24.0 => v1.29.0
go: upgraded go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.24.0 => v1.29.0
go: upgraded go.opentelemetry.io/proto/otlp v1.1.0 => v1.3.1
go: upgraded google.golang.org/genproto/googleapis/api v0.0.0-20240812133136-8ffd90a71988 => v0.0.0-20240822170219-fc7c04adadcd
go: upgraded google.golang.org/genproto/googleapis/rpc v0.0.0-20240812133136-8ffd90a71988 => v0.0.0-20240822170219-fc7c04adadcd

$ go get go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace
go: upgraded go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.49.0 => v0.54.0
go: upgraded go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.53.0 => v0.54.0

$ go get go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
go: upgraded go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.53.0 => v0.54.0

Now, running golangci-lint raises a few deprecation warnings, related to the google.golang.org/grpc upgrade:

$ make lint
internal/telemetry/traces.go:36:15: SA1019: grpc.DialContext is deprecated: use NewClient instead.  Will be supported throughout 1.x. (staticcheck)
    conn, err := grpc.DialContext(
                 ^
internal/telemetry/traces.go:40:3: SA1019: grpc.WithBlock is deprecated: this DialOption is not supported by NewClient. Will be supported throughout 1.x. (staticcheck)
        grpc.WithBlock(),
        ^
internal/a3m/client.go:21:15: SA1019: grpc.DialContext is deprecated: use NewClient instead.  Will be supported throughout 1.x. (staticcheck)
    conn, err := grpc.DialContext(
                 ^
make: *** [Makefile:179: lint] Error 1

I have changed the grpc.DialContext calls to use grpc.NewClient instead, but that doesn't accept a context or the WithBlock option. Some notes about those deprecation can be found in:

mcantelon commented 2 months ago

Nice! I'm in the process of trying it out! Did a go get -u ./... and go mod tidy and seemed to upgrade the go modules okay.

jraddaoui commented 2 months ago

Check https://enduro.readthedocs.io/dev-manual/deps/, @sevein suggested to avoid that.

jraddaoui commented 2 months ago

Thanks Mike! For the google.golang.org/grpc upgrade, I think I'll ignore the deprecation warning for now and create a new issue where we can discuss the changes better.