search

etcd-io / etcd

Distributed reliable key-value store for the most critical data of a distributed system
https://etcd.io
Apache License 2.0
47.83k stars 9.77k forks source link

Using KeepAliveOnce() will generate a new token every time #15764

Open yoonper opened 1 year ago

yoonper commented 1 year ago

What happened?

When I use KeepAliveOnce(), after 5 minutes, I can see deleted a simple token every 5 seconds in the log

What did you expect to happen?

No need to generate a new token every time KeepAliveOnce() is called

How can we reproduce it (as minimally and precisely as possible)?

package main

import (
    "context"
    clientv3 "go.etcd.io/etcd/client/v3"
    "google.golang.org/grpc"
    "time"
)

var (
    cli   *clientv3.Client
    ctx   = context.TODO()
    lease *clientv3.LeaseGrantResponse
)

func main() {
    cli, _ = clientv3.New(clientv3.Config{
        Endpoints:   []string{"xxx:2379"},
        DialTimeout: time.Second * 5,
        DialOptions: []grpc.DialOption{grpc.WithBlock()},
        Username:    "xxx",
        Password:    "xxx",
    })

    lease, _ = cli.Grant(ctx, 30)
    cli.Put(ctx, "/mykey", "", clientv3.WithLease(lease.ID))
    keepAlive()
}

func keepAlive() {
    ticker := time.NewTicker(time.Second * time.Duration(5))
    defer ticker.Stop()
    for range ticker.C {
        cli.KeepAliveOnce(ctx, lease.ID)
    }
}

Anything else we need to know?

go.etcd.io/etcd/client/v3 v3.5.8

Etcd version (please run commands below)

```console $ etcd --version etcd Version: 3.5.4 Git SHA: 08407ff76 Go Version: go1.16.15 Go OS/Arch: linux/amd64 $ etcdctl version etcdctl version: 3.5.4 API version: 3.5 ```

Etcd configuration (command line flags or environment variables)

/

Etcd debug information (please run commands below, feel free to obfuscate the IP address or FQDN in the output)

/

Relevant log output

{"level":"info","ts":"2023-04-24T16:32:05.161+0800","caller":"auth/simple_token.go:165","msg":"deleted a simple token","user-name":"xxx","token":"xxx"}
{"level":"info","ts":"2023-04-24T16:32:10.161+0800","caller":"auth/simple_token.go:165","msg":"deleted a simple token","user-name":"xxx","token":"xxx"}
{"level":"info","ts":"2023-04-24T16:32:15.161+0800","caller":"auth/simple_token.go:165","msg":"deleted a simple token","user-name":"xxx","token":"xxx"}
lavacat commented 1 year ago

This is normal behavior, etcd will check token of any request, not just KeepAliveOnce. From your logs I can infer that your --auth-token-ttl is 5s. So, if token > 5s, etcd will delete it.

yoonper commented 1 year ago

It doesn't seem like that. I adjusted the Tikcer time to 1s, and after 5 minutes, the token was being deleted every second

In addition, I used Get() instead of KeepAliveOnce(), and after 5 minutes, I haven't seen many logs of deleted a simple token

lavacat commented 1 year ago

the token was being deleted every second ... haven't seen many logs of deleted a simple token

Are you saying that Get behaves differently from KeepAliveOnce? Here the diff I'm using to reproduce(based on your code) and I see token deletions

diff --git a/Procfile b/Procfile
index 92ef37639..e97f1c593 100644
--- a/Procfile
+++ b/Procfile
@@ -1,9 +1,9 @@
 # Use goreman to run `go install github.com/mattn/goreman@latest`
 # Change the path of bin/etcd if etcd is located elsewhere

-etcd1: bin/etcd --name infra1 --listen-client-urls http://127.0.0.1:2379 --advertise-client-urls http://127.0.0.1:2379 --listen-peer-urls http://127.0.0.1:12380 --initial-advertise-peer-urls http://127.0.0.1:12380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof --logger=zap --log-outputs=stderr
-etcd2: bin/etcd --name infra2 --listen-client-urls http://127.0.0.1:22379 --advertise-client-urls http://127.0.0.1:22379 --listen-peer-urls http://127.0.0.1:22380 --initial-advertise-peer-urls http://127.0.0.1:22380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof --logger=zap --log-outputs=stderr
-etcd3: bin/etcd --name infra3 --listen-client-urls http://127.0.0.1:32379 --advertise-client-urls http://127.0.0.1:32379 --listen-peer-urls http://127.0.0.1:32380 --initial-advertise-peer-urls http://127.0.0.1:32380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof --logger=zap --log-outputs=stderr
+etcd1: bin/etcd --name infra1 --listen-client-urls http://127.0.0.1:2379 --advertise-client-urls http://127.0.0.1:2379 --listen-peer-urls http://127.0.0.1:12380 --initial-advertise-peer-urls http://127.0.0.1:12380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof --logger=zap --log-outputs=stderr --auth-token 'simple' --auth-token-ttl 1
+etcd2: bin/etcd --name infra2 --listen-client-urls http://127.0.0.1:22379 --advertise-client-urls http://127.0.0.1:22379 --listen-peer-urls http://127.0.0.1:22380 --initial-advertise-peer-urls http://127.0.0.1:22380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof --logger=zap --log-outputs=stderr --auth-token 'simple' --auth-token-ttl 1
+etcd3: bin/etcd --name infra3 --listen-client-urls http://127.0.0.1:32379 --advertise-client-urls http://127.0.0.1:32379 --listen-peer-urls http://127.0.0.1:32380 --initial-advertise-peer-urls http://127.0.0.1:32380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof --logger=zap --log-outputs=stderr --auth-token 'simple' --auth-token-ttl 1
 #proxy: bin/etcd grpc-proxy start --endpoints=127.0.0.1:2379,127.0.0.1:22379,127.0.0.1:32379 --listen-addr=127.0.0.1:23790 --advertise-client-url=127.0.0.1:23790 --enable-pprof

 # A learner node can be started using Procfile.learner
diff --git a/client/v3/client_test.go b/client/v3/client_test.go
index c569d7a3b..438f72947 100644
--- a/client/v3/client_test.go
+++ b/client/v3/client_test.go
@@ -18,6 +18,7 @@ import (
    "context"
    "errors"
    "fmt"
+
    "io"
    "net"
    "sync"
@@ -36,6 +37,25 @@ import (
    "google.golang.org/grpc"
 )

+func TestTest(t *testing.T) {
+   cli, _ := New(Config{
+       Endpoints:   []string{"127.0.0.1:2379"},
+       DialTimeout: time.Second * 5,
+       DialOptions: []grpc.DialOption{grpc.WithBlock()},
+       Username:    "root",
+       Password:    "root",
+   })
+   ctx := context.TODO()
+   lease, _ := cli.Grant(ctx, 30)
+   cli.Put(ctx, "/mykey", "", WithLease(lease.ID))
+
+   ticker := time.NewTicker(time.Second * time.Duration(5))
+   defer ticker.Stop()
+   for range ticker.C {
+       cli.Get(ctx, "/mykey")
+   }
+}
+
 func NewClient(t *testing.T, cfg Config) (*Client, error) {
    if cfg.Logger == nil {
        cfg.Logger = zaptest.NewLogger(t).Named("client")

Setup:

goreman -f Procfile start
./bin/etcdctl user add root
./bin/etcdctl --user root:root auth enable

Next I run TestTest in my IDE

yoonper commented 1 year ago

I did not set the option auth-token-ttl, KeepAliveOnce and Get perform differently in the same environment, only by using KeepAliveOnce have many tokens been deleted

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.