search

polyrabbit / etcdfs

:file_folder: Mount etcd into a local directory
MIT License
82 stars 7 forks source link

When used with SSL etcdfs throws error after a while #1

Open DicksengA opened 3 years ago

DicksengA commented 3 years ago

Able to mount with etcd client auth enabled. After approximately 3 mins, unable to interact with etcd. Error displays Failed to list keys from etcd, error="context deadline exceeded"

polyrabbit commented 3 years ago

This means a timeout error when reading from etcd. Could you try a larger timeout by specifying --read-timeout? maybe the default one (3s) is not enough in you case.

DicksengA commented 3 years ago

@polyrabbit Tried with 60s. Still getting the error. Weird thing is i works for a short duration of time before starting to display context deadline exceeded

polyrabbit commented 3 years ago

Could you add another parameter -v to get a more verbose log, and paste the whole log here?

DicksengA commented 3 years ago

INFO[04:39:54] Mounted to "/home/ec2-user/etcdfs", use ctrl+c to terminate. DEBU[04:40:02] Node Readdir path=/ DEBU[04:40:02] Node Lookup path=/ansible-playbook DEBU[04:40:02] Node Lookup path=/aws DEBU[04:40:02] Node Lookup path=/etcd-openshift DEBU[04:40:21] Node Readdir path=/ DEBU[04:40:21] Node Lookup path=/ansible-playbook DEBU[04:40:21] Node Lookup path=/aws DEBU[04:40:21] Node Lookup path=/etcd-openshift DEBU[04:40:24] Node Readdir path=/ DEBU[04:40:24] Node Lookup path=/ansible-playbook DEBU[04:40:24] Node Lookup path=/aws DEBU[04:40:24] Node Lookup path=/etcd-openshift DEBU[04:40:24] Node Readdir path=/ DEBU[04:40:24] Node Lookup path=/ansible-playbook DEBU[04:40:24] Node Lookup path=/aws DEBU[04:40:24] Node Lookup path=/etcd-openshift DEBU[04:40:24] Node Readdir path=/ DEBU[04:40:24] Node Lookup path=/ansible-playbook DEBU[04:40:24] Node Lookup path=/aws DEBU[04:40:24] Node Lookup path=/etcd-openshift DEBU[04:40:25] Node Lookup path=/ansible-playbook DEBU[04:40:26] Node Readdir path=/ansible-playbook/ DEBU[04:40:26] Node Lookup path=/ansible-playbook/etcd-server.yml DEBU[04:40:26] Node Lookup path=/ansible-playbook/setup-statemanager.yml DEBU[04:40:26] Node Lookup path=/ansible-playbook/ec2-baseline.yml DEBU[04:40:39] Node Lookup path=/ansible-playbook DEBU[04:40:40] Node Readdir path=/ DEBU[04:40:40] Node Lookup path=/ansible-playbook DEBU[04:40:40] Node Lookup path=/aws DEBU[04:40:40] Node Lookup path=/etcd-openshift DEBU[04:41:08] Node Readdir path=/ ERRO[04:41:18] Failed to list keys from etcd error="context deadline exceeded" path=/ DEBU[04:41:22] Node Readdir path=/ ERRO[04:41:32] Failed to list keys from etcd error="context deadline exceeded" path=/

xwjdsh commented 3 years ago

I have the same problem and I don't use SSL. Finally, I found out that the auto-sync of etcd caused it and if I commented the line AutoSyncInterval: time.Minute,, then everything works fine. I run etcd through docker, I‘m not sure if there is something wrong with my config.

polyrabbit commented 3 years ago

My etcd runs on a remote VM, and I don't have such an issue. How many entries do you have under that etcd prefix? One possible way to diagnose this issue is to open debug port and see what goroutines are doing via /debug/pprof/goroutine. Are there any pending connections that consuming too much resources?