flaky ci : test_watch_expired_events

[TCeason]

Summary

https://github.com/databendlabs/databend/actions/runs/12022226447/job/33514107559?pr=16936

--- STDOUT: databend-meta::it grpc::metasrv_grpc_watch::test_watch_expired_events ---

running 1 test test grpc::metasrv_grpc_watch::test_watch_expired_events ... FAILED

failures:

failures: grpc::metasrv_grpc_watch::test_watch_expired_events

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 62 filtered out; finished in 10.08s

--- STDERR: databend-meta::it grpc::metasrv_grpc_watch::test_watch_expired_events --- 2024-11-26T02:03:32.972060Z WARN it::tests::service: service.rs:198 new MetaSrvTestContext: can not remove raft_dir "/tmp/.tmpst94N8/29000/raft_dir", Os { code: 2, kind: NotFound, message: "No such file or directory" } 2024-11-26T02:03:32.998185Z WARN databend_common_meta_client::grpc_client: grpc_client.rs:548 MetaGrpcClient(127.0.0.1:29001) handle closed. worker quit 2024-11-26T02:03:43.019727Z WARN databend_common_meta_client::grpc_client: grpc_client.rs:548 MetaGrpcClient(127.0.0.1:29001) handle closed. worker quit thread 'grpc::metasrv_grpc_watch::test_watch_expired_events' panicked at src/meta/service/tests/it/grpc/metasrv_grpc_watch.rs:454:13: assertion left == right failed left: Some(Event { key: "w_b1", current: Some(SeqV { seq: 38, data: [119, 95, 98, 49, 95, 111, 118, 101, 114, 114, 105, 100, 101], meta: Some(KvMeta { expire_at: None }) }), prev: None }) right: Some(Event { key: "w_b3b", current: None, prev: Some(SeqV { seq: 37, data: [119, 95, 98, 51, 98], meta: Some(KvMeta { expire_at: Some(1732586000) }) }) }) note: run with RUST_BACKTRACE=1 environment variable to display a backtrace 2024-11-26T02:03:43.027968Z WARN databend_common_meta_client::grpc_client: grpc_client.rs:548 MetaGrpcClient(127.0.0.1:29001) handle closed. worker quit

[TCeason]

cc @drmingdrmer