Open clehner opened 3 years ago
I encountered this again today in local dev after pulling in a lot of new data. Indexing was stalling at 97% and the server wasn't fully booting and so wasn't responding to whoami.
After updating ssb-db to the latest version 20.4.0
with npm i -s ssb-db@latest
it seems to be indexing correctly and responding to commands again.
I did get some new errors in server logs that I hadn't seen before, so maybe they were causing the stall before:
could not retrive msg: Error [NotFoundError]: Key not found in database [@/3q5HcRfJu6KYA+zRmpwPEB5EN5QF8Ia4xhCrCTHtxw=.ed25519,1]
at /home/tim/projects/ssb-server/node_modules/levelup/lib/levelup.js:188:15
at /home/tim/projects/ssb-server/node_modules/encoding-down/index.js:75:21
could not retrive msg: Error [NotFoundError]: Key not found in database [@+sPe2KX1gMaYAiQJKTtBNlFT2bhZOHW07G7n1h4As+w=.ed25519,1]
at /home/tim/projects/ssb-server/node_modules/levelup/lib/levelup.js:188:15
at /home/tim/projects/ssb-server/node_modules/encoding-down/index.js:75:21
ssb-server as of v16 can get into an unresponsive state. In this state, it rejects incoming connections and does not index the flumelog.
When a secret-handshake client tries to connect to the server - even a local one - the server rejects the connection a few seconds after sending the first response packet. Example transcript from trying to connect to the server:
If the server is configured to use the unix noauth socket,
sbotc
can connect using that. (The-4
option used above forces it to use TCP and thus SHS instead of the noauth socket). Connecting with sbotc using the noauth socket reveals that ssb-server is not indexing its flumelog, as this command shows the progress stuck:It looks to me that ssb-server v16 gets into this state when it starts while the flumelog is not caught up. In previous versions, doing this would not be problem. ssb-server should allow appending data to the flumelog while it is offline and then indexing and catching up when it starts. Running ssb-server v15 causes it to catch up indexing, and then switching back to ssb-server v16 works again.
SSB thread reporting this issue:
%xPim4b5fwQ+3YDWkJ34WS2dN61gteHuu9UDqCY3Ipxg=.sha256