search

etcd-io / etcd

Distributed reliable key-value store for the most critical data of a distributed system
https://etcd.io
Apache License 2.0
47.64k stars 9.75k forks source link

clientv3: grpc-go (v1.27.0) made API changes to balancer / resolver. #11563

Closed eddycjy closed 4 years ago

eddycjy commented 4 years ago

After the release of grpc-go v1.27.0, an error occurred while pulling etcd / clientv3. The steps to reproduce it are as follows:

  1. go.mod:
go 1.13

require (
    github.com/coreos/etcd v3.3.18+incompatible // indirect
    github.com/coreos/go-systemd v0.0.0-00010101000000-000000000000 // indirect
    github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f // indirect
    github.com/gogo/protobuf v1.3.1 // indirect
    github.com/google/uuid v1.1.1 // indirect
    go.etcd.io/etcd v3.3.18+incompatible // indirect
    go.uber.org/zap v1.13.0 // indirect
    google.golang.org/grpc v1.27.0 // indirect
)

replace github.com/coreos/go-systemd => github.com/coreos/go-systemd/v22 v22.0.0
  1. command: 
$ go get go.etcd.io/etcd/clientv3
# github.com/coreos/etcd/clientv3/balancer/resolver/endpoint
../../go/pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:114:78: undefined: resolver.BuildOption
../../go/pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:182:31: undefined: resolver.ResolveNowOption
# github.com/coreos/etcd/clientv3/balancer/picker
../../go/pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/picker/err.go:37:44: undefined: balancer.PickOptions
../../go/pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/picker/roundrobin_balanced.go:55:54: undefined: balancer.PickOptions
  1. PR：https://github.com/etcd-io/etcd/pull/11564
xiang90 commented 4 years ago

/cc @jpbetz

nieaowei commented 4 years ago

版本不兼容，指定grpc的版本为1.26即可解决问题。 google.golang.org/grpc v1.26.0

eddycjy commented 4 years ago

@nieaowei Yes, but we need to solve the problem fundamentally.

g00nix commented 4 years ago

Did you find any workaround?

eddycjy commented 4 years ago

@gun1x You can first downgrade grpc to v1.26.0 to temporarily avoid this problem.

g00nix commented 4 years ago

@gun1x You can first downgrade grpc to v1.26.0 to temporarily avoid this problem.

ok, so the actual fix is adding this to go.mod:

require google.golang.org/grpc v1.26.0
replace github.com/coreos/go-systemd => github.com/coreos/go-systemd/v22 v22.0.0
mostfamiliar commented 4 years ago

@gun1x can i see your full go.mod for that fix? your suggestion does not fix the issue for me and i'm wondering if i'm missing something.


replace github.com/coreos/go-systemd => github.com/coreos/go-systemd/v22 v22.0.0

require (
    github.com/coreos/etcd v3.3.18+incompatible // indirect
    github.com/go-log/log v0.2.0 // indirect
    github.com/gogo/protobuf v1.3.1 // indirect
    github.com/json-iterator/go v1.1.9 // indirect
    github.com/lucas-clemente/quic-go v0.14.3 // indirect
    github.com/marten-seemann/qtls v0.7.1 // indirect
    github.com/micro/go-micro v1.18.0
    github.com/miekg/dns v1.1.27 // indirect
    github.com/mostfamiliar/shippy-service-consignment v0.0.0-20200212041600-94c659f72b79
    github.com/nats-io/jwt v0.3.2 // indirect
    github.com/pkg/errors v0.9.1 // indirect
    go.uber.org/atomic v1.5.1 // indirect
    go.uber.org/multierr v1.4.0 // indirect
    go.uber.org/zap v1.13.0 // indirect
    golang.org/x/crypto v0.0.0-20200210222208-86ce3cb69678 // indirect
    golang.org/x/lint v0.0.0-20200130185559-910be7a94367 // indirect
    golang.org/x/tools v0.0.0-20200211205636-11eff242d136 // indirect
    google.golang.org/genproto v0.0.0-20200211111953-2dc5924e3898 // indirect
    google.golang.org/grpc v1.27.1
)

error:

../../pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:114:78: undefined: resolver.BuildOption
../../pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:182:31: undefined: resolver.ResolveNowOption
# github.com/coreos/etcd/clientv3/balancer/picker
../../pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/picker/err.go:37:44: undefined: balancer.PickOptions
../../pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/picker/roundrobin_balanced.go:55:54: undefined: balancer.PickOptions
eddycjy commented 4 years ago

@mostfamiliar Your google.golang.org/grpc v1.27.1 should be modified to google.golang.org/grpc v1.26.0.

g00nix commented 4 years ago

@gun1x can i see your full go.mod for that fix? your suggestion does not fix the issue for me and i'm wondering if i'm missing something.

module mymod

go 1.13

require (
    github.com/coreos/etcd v3.3.18+incompatible
    github.com/coreos/go-systemd v0.0.0-00010101000000-000000000000 // indirect
    github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f // indirect
    github.com/gogo/protobuf v1.3.1 // indirect
    github.com/google/uuid v1.1.1 // indirect
    go.uber.org/zap v1.13.0 // indirect
    google.golang.org/grpc v1.26.0
)

replace github.com/coreos/go-systemd => github.com/coreos/go-systemd/v22 v22.0.0
arCzWw commented 4 years ago

Has this problem been solved?I have to use version 1.27.0 or I can't use micro, but I need to use etcd to get service information.

arges commented 4 years ago

I have a fix for this issue here: https://github.com/etcd-io/etcd/pull/11580. How do I get a review for it?

jbhannah commented 4 years ago

@arCzWw Micro still uses 1.26.0, so you should be able to pin to that version in your go.mod (this solved the issue for me).

yezihack commented 4 years ago

遇到同样的问题. meet the same problem

# github.com/coreos/etcd/clientv3/balancer/resolver/endpoint
vendor\github.com\coreos\etcd\clientv3\balancer\resolver\endpoint\endpoint.go:114:78: undefined: resolver.BuildOption
vendor\github.com\coreos\etcd\clientv3\balancer\resolver\endpoint\endpoint.go:182:31: undefined: resolver.ResolveNowOption
# github.com/coreos/etcd/clientv3/balancer/picker
vendor\github.com\coreos\etcd\clientv3\balancer\picker\err.go:37:44: undefined: balancer.PickOptions
vendor\github.com\coreos\etcd\clientv3\balancer\picker\roundrobin_balanced.go:55:54: undefined: balancer.PickOptions
13642212901 commented 4 years ago

if modified to google.golang.org/grpc v1.26.0... then protobuf build pb error undefined: grpc.SupportPackageIsVersion6..

eddycjy commented 4 years ago

@13642212901 The consideration is caused by the high version of protoc.

13642212901 commented 4 years ago

@13642212901 The consideration is caused by the high version of protoc.

I know. I hope etcd-io/etcd support grpc v1.27.*,it,s so easy.

hesongkang commented 4 years ago

I meet the same issue.

I use the latest version protobuf in https://github.com/protocolbuffers/protobuf/releases

and I go get -u github.com/golang/protobuf/protoc-gen-go

then if I modified to google.golang.org/grpc v1.26.0

protobuf build pb error undefined: grpc.SupportPackageIsVersion6.

if I modified to google.golang.org/grpc v1.27.0

then etcd will report err:

eddycjy commented 4 years ago

Workaround:

  1. If it is grpc> = 1.27.0, then just change grpc v1.26.0 in the go.mod file, as follows:

    require (
...
google.golang.org/grpc v1.26.0
)

  2. If the version of protoc is too high and grpc v1.26.0 is not compatible, you can downgrade protoc.

fix needs to wait for https://github.com/etcd-io/etcd/pull/11564.

yezihack commented 4 years ago

遇到同样的问题. meet the same problem

# github.com/coreos/etcd/clientv3/balancer/resolver/endpoint
vendor\github.com\coreos\etcd\clientv3\balancer\resolver\endpoint\endpoint.go:114:78: undefined: resolver.BuildOption
vendor\github.com\coreos\etcd\clientv3\balancer\resolver\endpoint\endpoint.go:182:31: undefined: resolver.ResolveNowOption
# github.com/coreos/etcd/clientv3/balancer/picker
vendor\github.com\coreos\etcd\clientv3\balancer\picker\err.go:37:44: undefined: balancer.PickOptions
vendor\github.com\coreos\etcd\clientv3\balancer\picker\roundrobin_balanced.go:55:54: undefined: balancer.PickOptions

resolve method: https://segmentfault.com/q/1010000021762281

OGKevin commented 4 years ago

Workaround:

  1. If it is grpc> = 1.27.0, then just change grpc v1.26.0 in the go.mod file, as follows:
require (
  ...
  google.golang.org/grpc v1.26.0
)
  1. If the version of protoc is too high and grpc v1.26.0 is not compatible, you can downgrade protoc.

fix needs to wait for #11564.

@eddycjy do you know which version of protoc is compatible ? Im stuck as i cant use 26 because of 

internal/storage/storage.pb.go:540:11: undefined: grpc.SupportPackageIsVersion6
internal/storage/storage.pb.go:551:5: undefined: grpc.ClientConnInterface

and cant use 27 because of this issue 😑 Any tips for other work arounds ?

OGKevin commented 4 years ago

@arCzWw Micro still uses 1.26.0, so you should be able to pin to that version in your go.mod (this solved the issue for me).

@jbhannah Unfortunately this is not the case for me. I get the issue internal/storage/storage.pb.go:540:11: undefined: grpc.SupportPackageIsVersion6 and according to the docs and code found on https://github.com/grpc/grpc-go, v1.27.0 introduced v6 and for v5 i indeed need to pin to v1.26.0. But still the generated code is using v6, i cant seem to get protoc to generate v5 code. Any pointers ?

jbhannah commented 4 years ago

@OGKevin Depends on the structure of your particular project, but I would remove the existing generated files and keep trying to get protoc to regenerate them from the protobuf definitions. I have this comment at the top of a Go file in the same directory as the .proto file and use Go's generate command in my workflow (I'm also using gogoproto):

//go:generate protoc --gogofaster_out=plugins=grpc:. --proto_path=$GOPATH/src:$GOPATH/pkg/mod:. api.proto
exceptionplayer commented 4 years ago

I have also encountered this problem, i have solved it with @gun1x's approach.

SoftTacos commented 4 years ago

@gun1x's approach didn't work for me, if it doesn't for others, try adding:

replace github.com/coreos/etcd v3.3.17+incompatible => github.com/coreos/etcd v3.3.4+incompatible

To your go.mod, didn't need to change grpc or proto versions

gogrisohil commented 4 years ago

Can a new release be created that can be used for the fix ?

phpstudyer commented 4 years ago

版本不兼容，指定grpc的版本为1.26即可解决问题。 google.golang.org/grpc v1.26.0

但是最新版的protoc编译出来的pb文件会提示undefined: grpc.SupportPackageIsVersion6 等问题

phpstudyer commented 4 years ago

遇到同样的问题. meet the same problem

# github.com/coreos/etcd/clientv3/balancer/resolver/endpoint
vendor\github.com\coreos\etcd\clientv3\balancer\resolver\endpoint\endpoint.go:114:78: undefined: resolver.BuildOption
vendor\github.com\coreos\etcd\clientv3\balancer\resolver\endpoint\endpoint.go:182:31: undefined: resolver.ResolveNowOption
# github.com/coreos/etcd/clientv3/balancer/picker
vendor\github.com\coreos\etcd\clientv3\balancer\picker\err.go:37:44: undefined: balancer.PickOptions
vendor\github.com\coreos\etcd\clientv3\balancer\picker\roundrobin_balanced.go:55:54: undefined: balancer.PickOptions

resolve method: https://segmentfault.com/q/1010000021762281

高版本protoc 一样抓瞎 protoc version 3.6.1

phpstudyer commented 4 years ago

github.com/coreos/etcd v3.3.17+incompatible => github.com/coreos/etcd v3.3.4+incompatible

good job ,thanks

cjb21sj commented 4 years ago

强制绑定grpc 1.26.0

module kratos-demo

go 1.13

require (
    github.com/bilibili/kratos v0.3.4-0.20200306133340-7c946dc919c1
    github.com/gogo/protobuf v1.3.1
    github.com/golang/protobuf v1.3.5
    github.com/google/wire v0.4.0
    github.com/pkg/errors v0.9.1 // indirect
    google.golang.org/genproto v0.0.0-20200318110522-7735f76e9fa5
    google.golang.org/grpc v1.27.0
)

replace (
    github.com/coreos/go-systemd => github.com/coreos/go-systemd/v22 v22.0.0
    google.golang.org/grpc => google.golang.org/grpc v1.26.0
)
r-shreesha commented 4 years ago

Workaround:

  1. If it is grpc> = 1.27.0, then just change grpc v1.26.0 in the go.mod file, as follows:
require (
    ...
    google.golang.org/grpc v1.26.0
)
  1. If the version of protoc is too high and grpc v1.26.0 is not compatible, you can downgrade protoc.

fix needs to wait for #11564.

@eddycjy do you know which version of protoc is compatible ? Im stuck as i cant use 26 because of

internal/storage/storage.pb.go:540:11: undefined: grpc.SupportPackageIsVersion6
internal/storage/storage.pb.go:551:5: undefined: grpc.ClientConnInterface

and cant use 27 because of this issue 😑 Any tips for other work arounds ?

@OGKevin Were you able to figure out a work around for this? I'm facing the same issue and looking for a fix.

OGKevin commented 4 years ago

@shreesha21 i removed the GRPC plugin in the protoc command and this seems to generate valid spec.

teone commented 4 years ago

is there any plan to upgrade etcd to support google.golang.org/grpc v1.26.0 and newer?

vooon commented 4 years ago

@teone i already use grpc 1.29.1, but you should use replace like that:

replace go.etcd.io/etcd => go.etcd.io/etcd v0.5.0-alpha.5.0.20200329194405-dd816f0735f8
neetuom commented 4 years ago

Hi I'm facing issue .

github.com/micro/go-micro/v2/registry/etcd ../../../pkg/mod/github.com/micro/go-micro/v2@v2.6.0/registry/etcd/etcd.go:79:10: config.LogConfig undefined (type clientv3.Config has no field or method LogConfig)

neetuom commented 4 years ago

module ship-microservice-vessel.com/ship-microservice-vessel

go 1.14

require ( github.com/golang/protobuf v1.4.1 github.com/grpc-ecosystem/grpc-gateway v1.14.5 github.com/micro/go-micro/v2 v2.6.0 google.golang.org/grpc v1.29.1 google.golang.org/protobuf v1.22.0 )

replace github.com/coreos/etcd v3.3.18+incompatible => github.com/coreos/etcd v3.3.4+incompatible

neetuom commented 4 years ago

github.com/coreos/etcd/clientv3/balancer/resolver/endpoint ../../../pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:114:78: undefined: resolver.BuildOption ../../../pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:182:31: undefined: resolver.ResolveNowOption

github.com/coreos/etcd/clientv3/balancer/picker

../../../pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/picker/err.go:37:44: undefined: balancer.PickOptions ../../../pkg/mod/github.com/coreos/etcd@v3.3.18+incompatible/clientv3/balancer/picker/roundrobin_balanced.go:55:54: undefined: balancer.PickOptions

above given issue is solved after downgrading the etcd version mentioned below github.com/coreos/etcd v3.3.18+incompatible => github.com/coreos/etcd v3.3.4+incompatible

neetuom commented 4 years ago

Please help to sort out the issue

invidian commented 4 years ago

@neetuom I recently managed to upgrade all my dependencies using the following set of replace statements, maybe that's helpful: https://github.com/flexkube/libflexkube/commit/f9cb8bda14eccbb2747eaac2c5007ef5d60d3939#diff-37aff102a57d3d7b797f152915a6dc16L85-R99

neetuom commented 4 years ago

@neetuom I recently managed to upgrade all my dependencies using the following set of replace statements, maybe that's helpful: flexkube/libflexkube@f9cb8bd#diff-37aff102a57d3d7b797f152915a6dc16L85-R99

Thanks Bro!

johnbellone commented 4 years ago

I'm still struggling here. Anyone have working example I can lurk?

For reference https://github.com/johnbellone/cache-service

refs commented 4 years ago

@invidian replacements did the job for me:

replace (
    github.com/coreos/etcd => go.etcd.io/etcd v0.5.0-alpha.5.0.20200425165423-262c93980547
    go.etcd.io/etcd => go.etcd.io/etcd v0.5.0-alpha.5.0.20200425165423-262c93980547
)

which is a little odd, since that branch is 6 years old.

johnbellone commented 4 years ago

I ended up nuking my attempt to get Bazel to work as a build environment and went back to simply using go mod and go build. I was able to quickly get it working with only the following:

go 1.14

require (
    github.com/golang/protobuf v1.4.2
    github.com/micro/go-micro/v2 v2.9.0
    google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1
)
xmlking commented 4 years ago

wish we move etcd dependency out of go-micro, maybe move it to plugins. this is giving bad experience for first-time go-micro users.

PVince81 commented 4 years ago

@invidian replacements did the job for me:

replace (
  github.com/coreos/etcd => go.etcd.io/etcd v0.5.0-alpha.5.0.20200425165423-262c93980547
  go.etcd.io/etcd => go.etcd.io/etcd v0.5.0-alpha.5.0.20200425165423-262c93980547
)

which is a little odd, since that branch is 6 years old.

doesn't seem to work, I get conflicts:

go: go.etcd.io/etcd@v0.5.0-alpha.5.0.20200425165423-262c93980547 used for two different module paths (github.com/coreos/etcd and go.etcd.io/etcd)
vtolstov commented 4 years ago

i'm use this trick - replace ( google.golang.org/grpc => google.golang.org/grpc v1.27.0 github.com/coreos/etcd => github.com/ozonru/etcd v3.3.20-grpc1.27-origmodule+incompatible )

refs commented 4 years ago

hmmmm, right. Thanks for sharing! I'll be using that one.

KyteProject commented 4 years ago

i'm use this trick - replace ( google.golang.org/grpc => google.golang.org/grpc v1.27.0 github.com/coreos/etcd => github.com/ozonru/etcd v3.3.20-grpc1.27-origmodule+incompatible )

Thanks mate, seems to have worked for me!

zxpan commented 4 years ago

Seems that etcd maintainer did not take serious to fix the issue. I still got this issue, where I do need latest gRPC (1.31+)

github.com/coreos/etcd/clientv3/balancer/resolver/endpoint

/home/alpha/go/pkg/mod/github.com/coreos/etcd@v3.3.25+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:114:78: undefined: resolver.BuildOption /home/alpha/go/pkg/mod/github.com/coreos/etcd@v3.3.25+incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:182:31: undefined: resolver.ResolveNowOption

github.com/coreos/etcd/clientv3/balancer/picker

/home/alpha/go/pkg/mod/github.com/coreos/etcd@v3.3.25+incompatible/clientv3/balancer/picker/err.go:37:44: undefined: balancer.PickOptions /home/alpha/go/pkg/mod/github.com/coreos/etcd@v3.3.25+incompatible/clientv3/balancer/picker/roundrobin_balanced.go:55:54: undefined: balancer.PickOptions

How I get the latest etcd golang module if there is a fix?

ptabor commented 4 years ago

@zxpan

The fix is in unreleased 3.5 branch: https://github.com/etcd-io/etcd/commit/4258cdd2efdf79a145067ae426f93b9f220c05f1#diff-c3bc35973ca604ca1c88d111806c06c4

and backported to 3.4: https://github.com/etcd-io/etcd/blob/release-3.4/clientv3/balancer/picker/roundrobin_balanced.go

Please try using etcd 3.4

clarkmcc commented 3 years ago

Why is this closed? I'm still having this issue and it looks like several others are as well.