search

BTrDB / btrdb-server

Berkeley Tree Database (BTrDB) server
http://btrdb.io
MIT License
909 stars 66 forks source link

Spontaneous down+out #44

Closed immesys closed 7 years ago

immesys commented 7 years ago

After a runtime of ~43 days, BTrDB spontaneously goes out

State changed: 
CLUSTER STATE e376
  LEADER: -- @0
  MASH: 0 (0 members, 100.00% unmapped)
  STATE: Degraded
  NODENAME  STATUS       MASH   WEIGHT RPREF
  btrdb-0   down+out     0      100    1.00 
ACTIVE MASH

PROPOSED MASH
[e38c8109]      btrdb-0 : 0 - 4294967296 (100)

$$ btrdb-0 allcurrent is false
$$ btrdb-0 proposedMash is 2, our target mash is 2
$$ btrdb-0 we do not want to advance our mash
immesys commented 7 years ago

This turns out to be a problem with leases and etcd. The solution might be to ensure that BTrDB is connected to all of the etcd EPs. We already added functionality so it will automatically restart if this occurs