Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.
I'm trying to use consul lock to execute a service on a single node but when that node is shutdown (to simulate a failure), the lock is taken on another node where there is a node check in critical state.
According to session documentation, the session should be invalidated if Any of the health checks go to the critical state but I do not have this behaviour.
Reproduction Steps
Steps to reproduce this issue, eg:
Create a cluster with 5 client/server nodes
Create a node check that is always critical on node2
Run on a first node consul lock -name simple-loop -verbose service/simple-loop/leader "/bin/bash -c 'while true; do echo "$(date --rfc-3339=seconds): OK"; sleep 5; done'", messages are displayed every 5 seconds
Run on a second node consul lock -name simple-loop -verbose service/simple-loop/leader "/bin/bash -c 'while true; do echo "$(date --rfc-3339=seconds): OK"; sleep 5; done'", the command is waiting on Attempting lock acquisition
On the first node, press Control-C
On the second node, messages are displayed every 5 seconds
Include appropriate Client or Server log fragments. If the log is longer than a few dozen lines, please include the URL to the gist of the log instead of posting it in the issue. Use -log-level=TRACE on the client and server to capture the maximum log detail.
Overview of the Issue
I'm trying to use
consul lock
to execute a service on a single node but when that node is shutdown (to simulate a failure), the lock is taken on another node where there is a node check in critical state.I tried to ask information on the forum.
According to session documentation, the session should be invalidated if
Any of the health checks go to the critical state
but I do not have this behaviour.Reproduction Steps
Steps to reproduce this issue, eg:
consul lock -name simple-loop -verbose service/simple-loop/leader "/bin/bash -c 'while true; do echo "$(date --rfc-3339=seconds): OK"; sleep 5; done'"
, messages are displayed every 5 secondsconsul lock -name simple-loop -verbose service/simple-loop/leader "/bin/bash -c 'while true; do echo "$(date --rfc-3339=seconds): OK"; sleep 5; done'"
, the command is waiting onAttempting lock acquisition
Control-C
Consul info for both Client and Server
Client info
``` agent: check_monitors = 0 check_ttls = 0 checks = 0 services = 0 build: prerelease = revision = bd257019 version = 1.14.3 version_metadata = consul: acl = disabled bootstrap = false known_datacenters = 1 leader = false leader_addr = 10.0.254.103:8300 server = true raft: applied_index = 347455 commit_index = 347455 fsm_pending = 0 last_contact = 90.181604ms last_log_index = 347455 last_log_term = 11 last_snapshot_index = 344141 last_snapshot_term = 9 latest_configuration = [{Suffrage:Voter ID:f6a3e39e-3f09-18cd-f753-e02d1a47022d Address:10.0.254.104:8300} {Suffrage:Voter ID:4acfff4c-891f-c212-af76-4ac7a6c3b735 Address:10.0.254.102:8300} {Suffrage:Voter ID:603d79f4-8408-9040-b2ad-0ed9566063a9 Address:10.0.254.101:8300} {Suffrage:Voter ID:8d5ab5dc-f221-ce2f-284e-6aaef2fadf31 Address:10.0.254.100:8300} {Suffrage:Voter ID:6e29d865-fa16-f9f0-d80d-7ec908ee6752 Address:10.0.254.103:8300}] latest_configuration_index = 0 num_peers = 4 protocol_version = 3 protocol_version_max = 3 protocol_version_min = 0 snapshot_version_max = 1 snapshot_version_min = 0 state = Follower term = 11 runtime: arch = amd64 cpu_count = 4 goroutines = 140 max_procs = 4 os = linux version = go1.19.4 serf_lan: coordinate_resets = 0 encrypted = true event_queue = 0 event_time = 9 failed = 0 health_score = 0 intent_queue = 0 left = 0 member_time = 39 members = 5 query_queue = 0 query_time = 1 serf_wan: coordinate_resets = 0 encrypted = true event_queue = 0 event_time = 1 failed = 0 health_score = 0 intent_queue = 0 left = 0 member_time = 43 members = 5 query_queue = 0 query_time = 1 ```Server info are the same.
Operating system and Environment details
OS, Architecture, and any other information you can provide about the environment.
5 identical Debian Bullseye virtual machines:
``` Distributor ID: Debian Description: Debian GNU/Linux 11 (bullseye) Release: 11 Codename: bullseye ```lsb_release -a
Log Fragments
Include appropriate Client or Server log fragments. If the log is longer than a few dozen lines, please include the URL to the gist of the log instead of posting it in the issue. Use
-log-level=TRACE
on the client and server to capture the maximum log detail.consul logs
``` janv. 30 15:38:29 moosefs-2 consul[536650]: 2023-01-30T15:38:29.222+0100 [WARN] agent: Check is now critical: check="Always fails" janv. 30 15:38:29 moosefs-2 consul[536650]: 2023-01-30T15:38:29.472+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:29 moosefs-2 consul[536650]: 2023-01-30T15:38:29.636+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:29 moosefs-2 consul[536650]: 2023-01-30T15:38:29.642+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:29 moosefs-2 consul[536650]: 2023-01-30T15:38:29.680+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:29 moosefs-2 consul[536650]: 2023-01-30T15:38:29.688+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:31 moosefs-2 consul[536650]: 2023-01-30T15:38:31.472+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:31 moosefs-2 consul[536650]: 2023-01-30T15:38:31.637+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:31 moosefs-2 consul[536650]: 2023-01-30T15:38:31.642+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:31 moosefs-2 consul[536650]: 2023-01-30T15:38:31.679+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:31 moosefs-2 consul[536650]: 2023-01-30T15:38:31.687+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:32 moosefs-2 consul[536650]: 2023-01-30T15:38:32.737+0100 [TRACE] agent.server: rpc_server_call: method=Session.Renew errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=false target_datacenter=dijon locality=local janv. 30 15:38:32 moosefs-2 consul[536650]: 2023-01-30T15:38:32.737+0100 [DEBUG] agent.http: Request finished: method=PUT url=/v1/session/renew/a3eb647d-596e-06fa-f99d-7774393590f9 from=127.0.0.1:44802 latency=1.270284ms janv. 30 15:38:33 moosefs-2 consul[536650]: 2023-01-30T15:38:33.386+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Get errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=true target_datacenter=dijon locality=local janv. 30 15:38:33 moosefs-2 consul[536650]: 2023-01-30T15:38:33.387+0100 [DEBUG] agent.http: Request finished: method=GET url="/v1/kv/service/moosefs/leader/.lock?index=347413&wait=15000ms" from=127.0.0.1:33182 latency=15.656219509s janv. 30 15:38:33 moosefs-2 consul[536650]: 2023-01-30T15:38:33.472+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:33 moosefs-2 consul[536650]: 2023-01-30T15:38:33.636+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:33 moosefs-2 consul[536650]: 2023-01-30T15:38:33.642+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:33 moosefs-2 consul[536650]: 2023-01-30T15:38:33.680+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:33 moosefs-2 consul[536650]: 2023-01-30T15:38:33.688+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:35 moosefs-2 consul[536650]: 2023-01-30T15:38:35.463+0100 [TRACE] agent.server.usage_metrics: Starting usage run janv. 30 15:38:35 moosefs-2 consul[536650]: 2023-01-30T15:38:35.472+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:35 moosefs-2 consul[536650]: 2023-01-30T15:38:35.636+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:35 moosefs-2 consul[536650]: 2023-01-30T15:38:35.642+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:35 moosefs-2 consul[536650]: 2023-01-30T15:38:35.679+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:35 moosefs-2 consul[536650]: 2023-01-30T15:38:35.688+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:37 moosefs-2 consul[536650]: 2023-01-30T15:38:37.472+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:37 moosefs-2 consul[536650]: 2023-01-30T15:38:37.637+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:37 moosefs-2 consul[536650]: 2023-01-30T15:38:37.642+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:37 moosefs-2 consul[536650]: 2023-01-30T15:38:37.680+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:37 moosefs-2 consul[536650]: 2023-01-30T15:38:37.688+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:39 moosefs-2 consul[536650]: 2023-01-30T15:38:39.227+0100 [TRACE] agent: Check output: check="Always fails" output="" janv. 30 15:38:39 moosefs-2 consul[536650]: 2023-01-30T15:38:39.227+0100 [WARN] agent: Check is now critical: check="Always fails" janv. 30 15:38:39 moosefs-2 consul[536650]: 2023-01-30T15:38:39.472+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:39 moosefs-2 consul[536650]: 2023-01-30T15:38:39.636+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:39 moosefs-2 consul[536650]: 2023-01-30T15:38:39.643+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:39 moosefs-2 consul[536650]: 2023-01-30T15:38:39.680+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:39 moosefs-2 consul[536650]: 2023-01-30T15:38:39.688+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:40 moosefs-2 consul[536650]: 2023-01-30T15:38:40.239+0100 [TRACE] agent.server: rpc_server_call: method=Session.Renew errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=false target_datacenter=dijon locality=local janv. 30 15:38:40 moosefs-2 consul[536650]: 2023-01-30T15:38:40.239+0100 [DEBUG] agent.http: Request finished: method=PUT url=/v1/session/renew/a3eb647d-596e-06fa-f99d-7774393590f9 from=127.0.0.1:44802 latency=1.376331ms janv. 30 15:38:40 moosefs-2 consul[536650]: 2023-01-30T15:38:40.839+0100 [DEBUG] agent.http: Request finished: method=GET url=/v1/agent/self from=127.0.0.1:59588 latency=1.019438ms janv. 30 15:38:40 moosefs-2 consul[536650]: 2023-01-30T15:38:40.849+0100 [TRACE] agent.server: rpc_server_call: method=Session.Apply errored=false request_type=write rpc_type=net/rpc leader=false target_datacenter=dijon locality=local janv. 30 15:38:40 moosefs-2 consul[536650]: 2023-01-30T15:38:40.849+0100 [DEBUG] agent.http: Request finished: method=PUT url=/v1/session/create from=127.0.0.1:59588 latency=8.170415ms janv. 30 15:38:40 moosefs-2 consul[536650]: 2023-01-30T15:38:40.850+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Get errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=false target_datacenter=dijon locality=local janv. 30 15:38:40 moosefs-2 consul[536650]: 2023-01-30T15:38:40.850+0100 [DEBUG] agent.http: Request finished: method=GET url=/v1/kv/service/simple-loop/leader/.lock?wait=15000ms from=127.0.0.1:59588 latency="425.397µs" janv. 30 15:38:41 moosefs-2 consul[536650]: 2023-01-30T15:38:41.471+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:41 moosefs-2 consul[536650]: 2023-01-30T15:38:41.636+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:41 moosefs-2 consul[536650]: 2023-01-30T15:38:41.642+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:41 moosefs-2 consul[536650]: 2023-01-30T15:38:41.681+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:41 moosefs-2 consul[536650]: 2023-01-30T15:38:41.688+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:43 moosefs-2 consul[536650]: 2023-01-30T15:38:43.472+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:43 moosefs-2 consul[536650]: 2023-01-30T15:38:43.636+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:43 moosefs-2 consul[536650]: 2023-01-30T15:38:43.643+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:43 moosefs-2 consul[536650]: 2023-01-30T15:38:43.680+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:43 moosefs-2 consul[536650]: 2023-01-30T15:38:43.687+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:44 moosefs-2 consul[536650]: 2023-01-30T15:38:44.463+0100 [TRACE] agent.server.usage_metrics: Starting usage run janv. 30 15:38:45 moosefs-2 consul[536650]: 2023-01-30T15:38:45.472+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:45 moosefs-2 consul[536650]: 2023-01-30T15:38:45.637+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:45 moosefs-2 consul[536650]: 2023-01-30T15:38:45.642+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:45 moosefs-2 consul[536650]: 2023-01-30T15:38:45.680+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:45 moosefs-2 consul[536650]: 2023-01-30T15:38:45.688+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:46 moosefs-2 consul[536650]: 2023-01-30T15:38:46.276+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Get errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=true target_datacenter=dijon locality=local janv. 30 15:38:46 moosefs-2 consul[536650]: 2023-01-30T15:38:46.276+0100 [DEBUG] agent.http: Request finished: method=GET url="/v1/kv/service/simple-loop/leader/.lock?index=347549&wait=15000ms" from=127.0.0.1:59588 latency=5.42554313s janv. 30 15:38:46 moosefs-2 consul[536650]: 2023-01-30T15:38:46.286+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Apply errored=false request_type=write rpc_type=net/rpc leader=false target_datacenter=dijon locality=local janv. 30 15:38:46 moosefs-2 consul[536650]: 2023-01-30T15:38:46.286+0100 [DEBUG] agent.http: Request finished: method=PUT url="/v1/kv/service/simple-loop/leader/.lock?acquire=cb51912b-c3bc-06c5-a6e1-3b6de0734aa6&flags=3304740253564472344" from=127.0.0.1:59588 latency=9.215896ms janv. 30 15:38:46 moosefs-2 consul[536650]: 2023-01-30T15:38:46.287+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Get errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=false target_datacenter=dijon locality=local janv. 30 15:38:46 moosefs-2 consul[536650]: 2023-01-30T15:38:46.287+0100 [DEBUG] agent.http: Request finished: method=GET url=/v1/kv/service/simple-loop/leader/.lock?consistent= from=127.0.0.1:59588 latency="681.303µs" janv. 30 15:38:47 moosefs-2 consul[536650]: 2023-01-30T15:38:47.376+0100 [TRACE] agent.server: rpc_server_call: method=Coordinate.Update errored=false request_type=write rpc_type=net/rpc leader=false target_datacenter=dijon locality=local janv. 30 15:38:47 moosefs-2 consul[536650]: 2023-01-30T15:38:47.472+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:47 moosefs-2 consul[536650]: 2023-01-30T15:38:47.637+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:47 moosefs-2 consul[536650]: 2023-01-30T15:38:47.643+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:47 moosefs-2 consul[536650]: 2023-01-30T15:38:47.680+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:47 moosefs-2 consul[536650]: 2023-01-30T15:38:47.688+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:47 moosefs-2 consul[536650]: 2023-01-30T15:38:47.742+0100 [TRACE] agent.server: rpc_server_call: method=Session.Renew errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=false target_datacenter=dijon locality=local janv. 30 15:38:47 moosefs-2 consul[536650]: 2023-01-30T15:38:47.742+0100 [DEBUG] agent.http: Request finished: method=PUT url=/v1/session/renew/a3eb647d-596e-06fa-f99d-7774393590f9 from=127.0.0.1:44802 latency=1.095246ms janv. 30 15:38:48 moosefs-2 consul[536650]: 2023-01-30T15:38:48.352+0100 [TRACE] agent.server: rpc_server_call: method=Session.Renew errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=false target_datacenter=dijon locality=local janv. 30 15:38:48 moosefs-2 consul[536650]: 2023-01-30T15:38:48.352+0100 [DEBUG] agent.http: Request finished: method=PUT url=/v1/session/renew/cb51912b-c3bc-06c5-a6e1-3b6de0734aa6 from=127.0.0.1:59592 latency="969.717µs" janv. 30 15:38:48 moosefs-2 consul[536650]: 2023-01-30T15:38:48.391+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Get errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=true target_datacenter=dijon locality=local janv. 30 15:38:48 moosefs-2 consul[536650]: 2023-01-30T15:38:48.391+0100 [DEBUG] agent.http: Request finished: method=GET url="/v1/kv/service/moosefs/leader/.lock?index=347413&wait=15000ms" from=127.0.0.1:33182 latency=15.003206691s janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.112+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Apply errored=false request_type=write rpc_type=net/rpc leader=false target_datacenter=dijon locality=local janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.112+0100 [DEBUG] agent.http: Request finished: method=PUT url="/v1/kv/service/simple-loop/leader/.lock?flags=3304740253564472344&release=cb51912b-c3bc-06c5-a6e1-3b6de0734aa6" from=127.0.0.1:59592 latency=9.333512ms janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.112+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Get errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=true target_datacenter=dijon locality=local janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.112+0100 [DEBUG] agent.http: Request finished: method=GET url="/v1/kv/service/simple-loop/leader/.lock?consistent=&index=347556" from=127.0.0.1:59588 latency=2.824764577s janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.113+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Get errored=false request_type=read rpc_type=net/rpc leader=false allow_stale=false blocking=false target_datacenter=dijon locality=local janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.113+0100 [DEBUG] agent.http: Request finished: method=GET url=/v1/kv/service/simple-loop/leader/.lock from=127.0.0.1:59592 latency="500.67µs" janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.121+0100 [TRACE] agent.server: rpc_server_call: method=Session.Apply errored=false request_type=write rpc_type=net/rpc leader=false target_datacenter=dijon locality=local janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.121+0100 [DEBUG] agent.http: Request finished: method=PUT url=/v1/session/destroy/cb51912b-c3bc-06c5-a6e1-3b6de0734aa6 from=127.0.0.1:59594 latency=8.289163ms janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.123+0100 [TRACE] agent.server: rpc_server_call: method=KVS.Apply errored=false request_type=write rpc_type=net/rpc leader=false target_datacenter=dijon locality=local janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.123+0100 [DEBUG] agent.http: Request finished: method=DELETE url=/v1/kv/service/simple-loop/leader/.lock?cas=347557 from=127.0.0.1:59592 latency=10.286325ms janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.231+0100 [TRACE] agent: Check output: check="Always fails" output="" janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.231+0100 [WARN] agent: Check is now critical: check="Always fails" janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.471+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.636+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.643+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.679+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false janv. 30 15:38:49 moosefs-2 consul[536650]: 2023-01-30T15:38:49.688+0100 [TRACE] agent.server: rpc_server_call: method=Status.RaftStats errored=false request_type=read rpc_type=net/rpc leader=false ```