possible bug: restart MQTT on `use-hd-keys`

stakwork / sphinx-relay

Node.js wrapper for communication between sphinx client and lightning node.

MIT License

250 stars 70 forks source link

Open Evanfeenstra opened 4 months ago

Evanfeenstra commented 4 months ago

We saw swarm4 relay crash when MQTT server reconnected (swarm4 is using use-hd-keys). Maybe there is a bug there?

Evanfeenstra commented 4 months ago

postmortem on the crash

MQTT was restarted (up, but missing env vars for sphinx-auth, which mqtt uses for authorization of new clients)
a couple hours later the EC2 was dead
big spike in EC2 EBS reads.... possibly relay trying over and over to hit MQTT and getting 401... and for some reason loading the sqlite again and again? Or maybe crashing and restarting over and over?