Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.
Consul binary is crashing when update from v1.3.0 to v1.5.2
Reproduction Steps
i really don't understand how problem occured, it was running and working fine from last 26 days in test environment, after that it started giving me issue:-
"
[root@bcmt1903-panch-control-02 ~]# kubectl logs ztsservicediscoveryserver-0
bootstrap_expect > 0: expecting 3 servers
==> Starting Consul agent...
==> Error starting agent: Failed to start Consul server: Failed to start Raft: failed to load any existing snapshots"
so i thought to update consul image .One thing i would say we do some KV r/w task in consul, are this can cause some problem like this?
Operating system and Environment details
consul latest docker image using "docker pull consul:latest"
2019/08/16 07:22:46 [DEBUG] tlsutil: Update with version 1
2019/08/16 07:22:46 [DEBUG] tlsutil: OutgoingRPCWrapper with version 1
2019/08/16 07:22:46 [INFO] raft: Restored from snapshot 145-1016371-1565779380540
2019/08/16 07:22:46 [ERROR] raft: Failed to get log at 1016372: log not found
Overview of the Issue
Consul binary is crashing when update from v1.3.0 to v1.5.2
Reproduction Steps
i really don't understand how problem occured, it was running and working fine from last 26 days in test environment, after that it started giving me issue:- " [root@bcmt1903-panch-control-02 ~]# kubectl logs ztsservicediscoveryserver-0
bootstrap_expect > 0: expecting 3 servers
==> Starting Consul agent...
==> Error starting agent: Failed to start Consul server: Failed to start Raft: failed to load any existing snapshots" so i thought to update consul image .One thing i would say we do some KV r/w task in consul, are this can cause some problem like this?
Operating system and Environment details
consul latest docker image using "docker pull consul:latest"
Log Fragments
ztsservicediscoveryserver-0 0/1 CrashLoopBackOff 3 104s
ztsservicediscoveryserver-1 0/1 CrashLoopBackOff 28 120m
ztsservicediscoveryserver-2 0/1 CrashLoopBackOff 165 13h
[root@bcmt1903-panch-control-02 ~]# kubectl logs ztsservicediscoveryserver-0
bootstrap_expect > 0: expecting 3 servers
==> Starting Consul agent...
==> Log data will now stream in as it occurs:
panic: log not found
goroutine 1 [running]:
github.com/hashicorp/raft.NewRaft(0xc0004c0000, 0x32d4340, 0xc000726c60, 0x330ad80, 0xc00043c840, 0x32ee680, 0xc000366c40, 0x32d6000, 0xc000548000, 0x3326700, ...)
github.com/hashicorp/consul/agent/consul.(*Server).setupRaft(0xc0004b0700, 0x0, 0x0)
github.com/hashicorp/consul/agent/consul.NewServerLogger(0xc0004b0380, 0xc000315860, 0xc0004f6080, 0xc000102050, 0x0, 0x0, 0x0)
github.com/hashicorp/consul/agent.(*Agent).Start(0xc00011a480, 0x0, 0x0)
github.com/hashicorp/consul/command/agent.(*cmd).run(0xc000176000, 0xc0000fe020, 0x10, 0x10, 0x0)
github.com/hashicorp/consul/command/agent.(*cmd).Run(0xc000176000, 0xc0000fe020, 0x10, 0x10, 0xc0001b5b40)
github.com/mitchellh/cli.(*CLI).Run(0xc000258140, 0xc000258140, 0x80, 0xc0000ce840)
main.realMain(0xc0000c2058)
main.main()