Closed shnewto closed 1 year ago
It turns out there was a mistake in the release process in the move to using Actions to do parts of it. That was tracked and fixed in https://github.com/apache/beam/issues/27064 as a result there's now a v2.48.2 patch release for the Go SDK, to resolve that issue.
It will still use the 2.48.0 containers by default however, since those were not broken, and were not rebuilt and tagged as a part of this process.
Ironically, due to how Go Modules versions are pushed, there's no test in the world that would have caught this before the tag, since the RC tags were correct. The problem was the RC tag wasn't used in making the Go SDK release tag!
So, @shnewto I'll be closing this issue, since I think the v2.48.2 patch should resolve this for you. But please do reopen if it is causing issues with flex templates.
What needs to happen?
Hello! 👋
I notice that the GO SDK defines its version as
.dev
of whatever version it's on,SdkVersion = "2.49.0.dev"
currently but it also has the.dev
in the actual releases, i.e.SdkVersion = "2.48.0.dev"
for the2.48.0
release https://github.com/apache/beam/blob/425186f27dc2eef0e2e45b23cd6eeb7d01ceef20/sdks/go/pkg/beam/core/core.go#L30 (This may be a bug actually, but looking through the file's history it didn't look like un-qualified version numbers were used recently.)I bring it up because it makes things sort of tricky to use the SDK in base Dataflow Flex templates in GCP. Google says it doesn't support
.dev
version numbers in Flex templates so when Dataflow jobs are triggered, they fail because Google is looking for2.48.0.dev
(unsupported) instead of2.48.0
. I have a couple workarounds, one is to vendor the source code and change theSdkVersion
. The other that seems to work is to passenvironment_config='apache/beam_go_sdk:2.48.0
to short circuit whatever's happening in thegcr.io/dataflow-templates-base/go-template-launcher-base
image.I'd be happy to make a PR if there's an appropriate thing to change in docs or some pre-release tests? But it seems like this might just be a process thing for the next
2.49.0
release.Thanks for all the work that goes into Beam!
*EDIT: I forgot to even mention that the SDK version linked above is being used to define
DefaultDockerImage
which is actually what's making GCP hiccup I think https://github.com/apache/beam/blob/425186f27dc2eef0e2e45b23cd6eeb7d01ceef20/sdks/go/pkg/beam/core/core.go#L33Issue Priority
Priority: 2 (default / most normal work should be filed as P2)
Issue Components