etcd-io / zetcd

Serve the Apache Zookeeper API but back it with an etcd cluster
Apache License 2.0
1.09k stars 94 forks source link

Does not seem to work on Windows 10 #80

Open Jeff17Robbins opened 7 years ago

Jeff17Robbins commented 7 years ago

I downloaded etcd-v3.2.5-windows-amd64 from Releases and ran etcd.exe.

I then followed the instructions to build and run zetcd:

go get github.com/coreos/zetcd/cmd/zetcd
zetcd --zkaddr 0.0.0.0:2181 --endpoints localhost:2379

Output of zetcd:

C:\Users\jeffr\go\bin>zetcd --zkaddr 0.0.0.0:2181 --endpoints localhost:2379
Running zetcd proxy
Version: Version not provided (use make instead of go build)
SHA: SHA not provided (use make instead of go build)

I then ran the zkctl simple test:

go get github.com/coreos/zetcd/cmd/zkctl
zkctl watch / &
zkctl create /abc "foo"

The zkctl watch / command produced this output:

C:\Users\jeffr\go\bin>zkctl watch /
watch dir /
2017/08/10 14:50:16 Connected to 127.0.0.1:2181
2017/08/10 14:50:16 Authenticated: id=7587824084266938116, timeout=1000
[] &{Czxid:0 Mzxid:0 Ctime:0 Mtime:0 Version:0 Cversion:-1 Aversion:0 EphemeralOwner:0 DataLength:0 NumChildren:1 Pzxid:0}

The zkctl create /abc "foo" command produced this output:

C:\Users\jeffr\go\bin>zkctl create /abc "foo"
2017/08/10 14:50:43 Connected to 127.0.0.1:2181
2017/08/10 14:50:43 Authenticated: id=7587824084266938119, timeout=1000

But then the zkctl watch / command seemed to have a problem and quit running (I'm assuming this is unexpected output?):

{Type:EventNodeChildrenChanged State:Unknown Path:/ Err:<nil> Server:}

I am completely new to both etcd and zetcd, so I apologize in advance for not knowing what I am doing. Ought this example work?

My go version is:

go version go1.8.3 windows/amd64

image

2017-08-10 14:50:16.678468 W | etcdserver: apply entries took too long [408.5913ms for 1 entries] 2017-08-10 14:50:16.680467 W | etcdserver: avoid queries with large range/delete range! 2017-08-10 14:50:43.719299 W | etcdserver: apply entries took too long [256.5433ms for 1 entries] 2017-08-10 14:50:43.719299 W | etcdserver: avoid queries with large range/delete range! 2017-08-10 14:50:45.520147 W | etcdserver: apply entries took too long [158.998ms for 1 entries] 2017-08-10 14:50:45.521148 W | etcdserver: avoid queries with large range/delete range!

heyitsanthony commented 7 years ago

Compared against a regular zk node, works as intended:

Launched zk:

$ docker run --net host --rm zetcd-zk-test
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/conf/zoo.cfg
2017-08-10 19:12:20,422 [myid:] - INFO  [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper/conf/zoo.cfg
2017-08-10 19:12:20,428 [myid:] - INFO  [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2017-08-10 19:12:20,428 [myid:] - INFO  [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2017-08-10 19:12:20,428 [myid:] - INFO  [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2017-08-10 19:12:20,429 [myid:] - WARN  [main:QuorumPeerMain@113] - Either no config or no quorum defined in config, running  in standalone mode
2017-08-10 19:12:20,443 [myid:] - INFO  [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper/conf/zoo.cfg
2017-08-10 19:12:20,443 [myid:] - INFO  [main:ZooKeeperServerMain@95] - Starting server
2017-08-10 19:12:20,451 [myid:] - INFO  [main:Environment@100] - Server environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT
2017-08-10 19:12:20,451 [myid:] - INFO  [main:Environment@100] - Server environment:host.name=cc0db6606a67
2017-08-10 19:12:20,451 [myid:] - INFO  [main:Environment@100] - Server environment:java.version=1.8.0_111-internal
2017-08-10 19:12:20,451 [myid:] - INFO  [main:Environment@100] - Server environment:java.vendor=Oracle Corporation
2017-08-10 19:12:20,452 [myid:] - INFO  [main:Environment@100] - Server environment:java.home=/usr/lib/jvm/java-1.8-openjdk/jre
2017-08-10 19:12:20,452 [myid:] - INFO  [main:Environment@100] - Server environment:java.class.path=/opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/opt/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/zookeeper/bin/../zookeeper-3.4.8.jar:/opt/zookeeper/bin/../src/java/lib/*.jar:/opt/zookeeper/bin/../conf:
2017-08-10 19:12:20,452 [myid:] - INFO  [main:Environment@100] - Server environment:java.library.path=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-08-10 19:12:20,452 [myid:] - INFO  [main:Environment@100] - Server environment:java.io.tmpdir=/tmp
2017-08-10 19:12:20,452 [myid:] - INFO  [main:Environment@100] - Server environment:java.compiler=<NA>
2017-08-10 19:12:20,454 [myid:] - INFO  [main:Environment@100] - Server environment:os.name=Linux
2017-08-10 19:12:20,454 [myid:] - INFO  [main:Environment@100] - Server environment:os.arch=amd64
2017-08-10 19:12:20,454 [myid:] - INFO  [main:Environment@100] - Server environment:os.version=4.11.9
2017-08-10 19:12:20,454 [myid:] - INFO  [main:Environment@100] - Server environment:user.name=root
2017-08-10 19:12:20,454 [myid:] - INFO  [main:Environment@100] - Server environment:user.home=/root
2017-08-10 19:12:20,454 [myid:] - INFO  [main:Environment@100] - Server environment:user.dir=/opt/zookeeper
2017-08-10 19:12:20,464 [myid:] - INFO  [main:ZooKeeperServer@787] - tickTime set to 2000
2017-08-10 19:12:20,465 [myid:] - INFO  [main:ZooKeeperServer@796] - minSessionTimeout set to -1
2017-08-10 19:12:20,465 [myid:] - INFO  [main:ZooKeeperServer@805] - maxSessionTimeout set to -1
2017-08-10 19:12:20,501 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181

Started watch:

 $ ./zkctl watch /
watch dir /
2017/08/10 12:13:33 Connected to 127.0.0.1:2181
2017/08/10 12:13:34 Authenticated: id=98460789108375552, timeout=4000
[zookeeper] &{Czxid:0 Mzxid:0 Ctime:0 Mtime:0 Version:0 Cversion:-1 Aversion:0 EphemeralOwner:0 DataLength:0 NumChildren:1 Pzxid:0}

Created key:

$ ./zkctl create /abc def
2017/08/10 12:14:36 Connected to 127.0.0.1:2181
2017/08/10 12:14:36 Authenticated: id=98460789108375553, timeout=4000

Watch received as expected

{Type:EventNodeChildrenChanged State:Unknown Path:/ Err:<nil> Server:}

The watches are one shot in zkctl (https://github.com/coreos/zetcd/blob/master/cmd/zkctl/zkctl.go#L121) so it will exit on receiving an event. The command uses a children watch, so it only receives create/delete (EventNodeChildrenChanged) but not set. Sorry it's so cryptic, the output/documentation/behavior could use some improvement. Maybe the commands should be more similar to zkCli?

The etcd apply warnings are usually because etcd can't persist data fast enough; typically something else on the system is eating cpu/disk bandwidth.

Jeff17Robbins commented 7 years ago

Oh, my bad, sorry!

According to this kubernetes issue,

Any LIST query will result in this log line in etcd:

etcdserver: avoid queries with large range/delete range! 

This is kind of working-as-intended.

I am guessing that zetcd uses a LIST query?

heyitsanthony commented 7 years ago

No, list is a k8s only thing. It happens whenever an rpc takes too long in the raft path.

I don't think it's related to zetcd. Try go get github.com/coreos/etcd/cmd/benchmark and see how etcd on that machine compares to the benchmarks at https://github.com/coreos/etcd/blob/master/Documentation/op-guide/performance.md#benchmarks

Jeff17Robbins commented 7 years ago

sorry, more newbie floundering:

C:\Users\jeffr\go>go get github.com/coreos/etcd/cmd/benchmark package github.com/coreos/etcd/cmd/benchmark: cannot find package " github.com/coreos/etcd/cmd/benchmark" in any of: C:\Go\src\github.com\coreos\etcd\cmd\benchmark (from $GOROOT) c:\Users\jeffr\go\src\github.com\coreos\etcd\cmd\benchmark (from $GOPATH)

On Thu, Aug 10, 2017 at 3:56 PM Anthony Romano notifications@github.com wrote:

No, list is a k8s only thing. It happens whenever an rpc takes too long in the raft path.

I don't think it's related to zetcd. Try go get github.com/coreos/etcd/cmd/benchmark and see how etcd on that machine compares to the benchmarks at https://github.com/coreos/etcd/blob/master/Documentation/op-guide/performance.md#benchmarks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/coreos/zetcd/issues/80#issuecomment-321656732, or mute the thread https://github.com/notifications/unsubscribe-auth/ACAWhwoOwJ-XnbaNLNiIeceVCJ3OU5KBks5sW2B2gaJpZM4Oz2I4 .

heyitsanthony commented 7 years ago

Oops, go get github.com/coreos/etcd/cmd/tools/benchmark; not sure if it will build that way on windows since some parts depend on junctions that are set up by build.bat.

Jeff17Robbins commented 7 years ago

That failed too. I'm too new at this to be all that helpful, sorry about that. My main interest in zetcd is to run Kafka without Zookeeper. I'm hoping I have things working well enough to start trying that.

On Thu, Aug 10, 2017 at 4:30 PM Anthony Romano notifications@github.com wrote:

Oops, go get github.com/coreos/etcd/cmd/tools/benchmark; not sure if it will build that way on windows since some parts depend on junctions that are set up by build.bat.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/coreos/zetcd/issues/80#issuecomment-321664766, or mute the thread https://github.com/notifications/unsubscribe-auth/ACAWh2sEOMWlchQhaEodLQmM9UPm9ltYks5sW2h6gaJpZM4Oz2I4 .