zeromicro / go-zero

A cloud-native Go microservices framework with cli tool for productivity.
https://go-zero.dev
MIT License
28.99k stars 3.92k forks source link

go-zero etcd invalid auth token, very serious #4393

Open iyyzh opened 5 days ago

iyyzh commented 5 days ago

For versions later than gozero 1.6.1, when the program is running normally, if etcd is turned off, this problem will occur when it is restarted after a few minutes.

{"@timestamp":"2024-05-22T00:40:14.826-03:00","caller":"internal/registry.go:305","content":"etcd monitor chan has been canceled, error: rpc error: code = Unauthenticated desc = etcdserver: invalid auth token","level":"error"}

The server crashed after hundreds of logs were printed out every second. The root cause was that I parsed the token and found that the old token was still used after etcd was restarted. This problem did not occur before gozero 1.6.1.

{"level":"warn","ts":"2024-09-03T17:24:45.757306+0800","caller":"auth/jwt.go:63","msg":"failed to parse a JWT token","token":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MjUzNTAwOTMsInJldmlzaW9uIjozLCJ1c2VybmFtZSI6InJvb3QifQ.Eyr2tVzQeRyslumR5hqBXzSrXPccDozEKPlYtf7_BaR92WxxVvq0eTHwLD_j4Xd6h-yiIfPuaAxnjUCbB26l8QeNiZNjDcIOoxY9Q2pucwUutvXDy-QwA33ir1g88T1CIiik297vC9PIrzouQPEKNM0RyVcf5PHrizqXWXXISiRRy78Ky9mjW-hfJST6GCnWGjUFkBbAs8sbsWRObN1871HTIpGkesnAXdBuKDG7B7kSCxSLquw6FbeDCFU03Z4sqqUg9DLNvtFUJ33BxxB2zJiOkgrnTtdD11yq6JG0eKsDkj4rkRdZayx5y_zlx0cINnbaGjtk_RKhie01HtHCFBaGMBI5WwtGmLk-uGVlsBbZBqKpsMAzDXzNge7AQH3e-lua3xnE2UYvnXlyD8AuaPASSEPl-crSq0VLTebSYDOhN7MW6iLKjGktIIXRIPcHasohZQQ3tmm2zxWUprOO573OUSfHkVddlK_3ywYD6BXE4zTjPfgXFKX-4YGqMyAEFP7MOY8hgWxkQKU2j_h3QGnXMa66evAJ6X6IyRaMrsjWNY85xKOHUMpaeA2n7NLkR28fzehLYh4ZwE6RnQ2m9xpE7xNYI7lO8vuHDVBfiRMwj-KiCAeeRfjPfVnWGeCMZt7WnBsIga0TPng3L1zhH2UQJpNX4JHyZe1OvcXoPB4","error":"Token is expired"}

For example, the time of this token parsing is: 2024-09-03 15:54:53, but the actual server time is 2024-09-03T17:24:45.757306+0800

Issues-translate-bot commented 5 days ago

Bot detected the issue body's language is not English, translate it automatically. πŸ‘―πŸ‘­πŸ»πŸ§‘β€πŸ€β€πŸ§‘πŸ‘«πŸ§‘πŸΏβ€πŸ€β€πŸ§‘πŸ»πŸ‘©πŸΎβ€πŸ€β€πŸ‘¨πŸΏπŸ‘¬πŸΏ


etcd is a cluster

iyyzh commented 4 days ago

https://github.com/etcd-io/etcd/issues/12385

https://wxsm.space/2024/etcd-queue-problem/

References, but why is go-zero version 1.6.1 fine?