saltstack-formulas / opensds-formula

Integration with Open Software Defined Storage - Capri release - SuSE/CentOS/Ubuntu
https://opensds.io
Other
4 stars 2 forks source link

osdsdock - Error: context deadline exceeded #25

Closed noelmcloughlin closed 5 years ago

noelmcloughlin commented 5 years ago

Seems https://github.com/etcd-io/etcd/issues/8793 is causing osdsdock container failures.

    /root/gopath/src/github.com/opensds/opensds/cmd/osdsdock/osdsdock.go:51 +0x70
E1226 00:28:06.994061       1 utils.go:100] Get etcd client:context deadline exceeded, retry 1 time(s)
E1226 00:28:09.995494       1 utils.go:100] Get etcd client:context deadline exceeded, retry 2 time(s)
E1226 00:28:12.996046       1 utils.go:100] Get etcd client:context deadline exceeded, retry 3 time(s)
E1226 00:28:12.996095       1 utils.go:107] Get etcd client retry exceed the max retry times(3).
panic: context deadline exceeded

goroutine 1 [running]:
github.com/opensds/opensds/pkg/db/drivers/etcd.Init(0xc00000c060, 0x2, 0x2, 0x2)
    /root/gopath/src/github.com/opensds/opensds/pkg/db/drivers/etcd/client.go:70 +0x110
github.com/opensds/opensds/pkg/db/drivers/etcd.NewClient(0xc00000c060, 0x2, 0x2, 0x1)
    /root/gopath/src/github.com/opensds/opensds/pkg/db/drivers/etcd/etcd.go:64 +0x43
github.com/opensds/opensds/pkg/db.Init(0xc000137070)
    /root/gopath/src/github.com/opensds/opensds/pkg/db/db.go:45 +0x1a6
main.main()
    /root/gopath/src/github.com/opensds/opensds/cmd/osdsdock/osdsdock.go:51 +0x70
noelmcloughlin commented 5 years ago

We could disable TLS: https://github.com/etcd-io/etcd/issues/8270 or enable certs on docker clients: https://pcocc.readthedocs.io/en/latest/deps/etcd-production.html

noelmcloughlin commented 5 years ago

Fixed by #34