search

gluster / glusterd2

[DEPRECATED] Glusterd2 is the distributed management framework to be used for GlusterFS.
GNU General Public License v2.0
167 stars 82 forks source link

glusterd2 crashes ~40 seconds after starting complaining "etcd took long to start" #1007

Open sac opened 6 years ago

sac commented 6 years ago

glusterd crashes with the following log message:

time="2018-07-12 17:27:51.411869" level=debug msg="Starting GlusterD" pid=23337 source="[main.go:70:main.main]" version=v4.1.0-59.git2b38ce4
time="2018-07-12 17:27:51.412058" level=info msg="loaded configuration from file" file=glusterd2.toml source="[config.go:106:main.dumpConfigToLog]"
time="2018-07-12 17:27:51.412588" level=debug msg="running with configuration" cert-file= clientaddress=":24007" config=glusterd2.toml defaultpeerport=24008 etcd-client-ca-file= etcd-client-cert-file= etcd-client-key-file= etcdcurls="http://10.70.43.250:2379" etcdendpoints="[]" etcdpurls="http://10.70.43.250:2380" hooksdir=/usr/local/var/lib/glusterd2/hooks key-file= localstatedir=/usr/local/var/lib/glusterd2 logdir=/usr/local/var/log/glusterd2 logfile=glusterd2.log loglevel=debug noembed=false peeraddress="10.70.43.250:24008" pidfile=/usr/local/var/run/glusterd2/glusterd2.pid rundir=/usr/local/var/run/glusterd2 source="[config.go:115:main.dumpConfigToLog]" statedump=true version=false
time="2018-07-12 17:27:51.420358" level=debug msg="Xlators dir found" source="[load.go:166:xlator.loadAllXlators]" xlatordir=/usr/local/lib/glusterfs/4.2dev/xlator
time="2018-07-12 17:27:51.431395" level=info msg="Registered validation function for xlator" source="[load.go:129:xlator.loadXlator]" xlator=afr
time="2018-07-12 17:27:51.431518" level=debug msg="Registered option actor for xlator" source="[load.go:135:xlator.loadXlator]" xlator=afr
time="2018-07-12 17:27:51.433550" level=info msg="Registered validation function for xlator" source="[load.go:129:xlator.loadXlator]" xlator=dht
time="2018-07-12 17:27:51.437414" level=info msg="Registered validation function for xlator" source="[load.go:129:xlator.loadXlator]" xlator=distribute
time="2018-07-12 17:27:51.443302" level=info msg="Registered validation function for xlator" source="[load.go:129:xlator.loadXlator]" xlator=replicate
time="2018-07-12 17:27:51.443393" level=debug msg="Registered option actor for xlator" source="[load.go:135:xlator.loadXlator]" xlator=replicate
time="2018-07-12 17:27:51.509218" level=info msg="Registered validation function for xlator" source="[load.go:129:xlator.loadXlator]" xlator=quota
time="2018-07-12 17:27:51.509319" level=debug msg="Registered option actor for xlator" source="[load.go:135:xlator.loadXlator]" xlator=quota
time="2018-07-12 17:27:51.571771" level=debug msg="saving updated store config" source="[config.go:179:store.GetConfig]"
time="2018-07-12 17:27:51.572442" level=debug msg="starting embedded store" cafile= certfile= curls="http://10.70.43.250:2379" datadir=/usr/local/var/lib/glusterd2/store endpoints="http://localhost:2379" keyfile= logdir=/usr/local/var/log/glusterd2/store name=de46f1ce-0809-4f8d-aefa-cda1c84f9ee3 purls="http://10.70.43.250:2380" source="[embed.go:32:store.newEmbedStore]" trustedcafile=
time="2018-07-12 17:28:33.805022" level=error msg="failed to start embedded store" error="etcd embedded server took too long to start" source="[embed.go:36:store.newEmbedStore]"
time="2018-07-12 17:28:33.805163" level=fatal msg="Failed to initialize store (etcd client)" error="etcd embedded server took too long to start" source="[main.go:100:main.main]"

glusterd version: 

[root@dhcp43-250 build]# ./glusterd2 --version
glusterd version: v4.1.0-59.git2b38ce4
git SHA: 2b38ce4
go version: go1.10.3
go OS/arch: linux/amd64
[root@dhcp43-250 build]#

What is the reason for the etcd embedded server to fail?

atinmu commented 6 years ago

@sac Was this done with embedded etcd mode?

sac commented 6 years ago

@atinmu Yes it is embeded etcd.