search

uaktags / explorer

An open source block explorer
BSD 3-Clause "New" or "Revised" License
6 stars 8 forks source link

Test Needed - Load Balance CoinDaemons / RoundRobin CoinDaemons #26

Closed uaktags closed 5 years ago

uaktags commented 5 years ago

So one thing that I was thinking of, which has nothing to do with the code/project of the explorer, but I wonder how stability would be handled if there were multiple coin daemons that the workers could communicate with.

The thought process would be to either extend the settings.json to have multiple wallets, with an order of priority setup. Then the users can either set multiple guys and we'll just go in order as workers are being created (roundrobin) and use each daemon. Alternatively, test with having a loadbalancer in the mix to route traffic to a wallet. This would be perfect for those that are on public webhosts that offer such functionality (like ionos).

uaktags commented 5 years ago

So running with HaProxy is a stupid simple setup to get 2 or more daemons online to cluster the api. I no longer get any issues with the daemon crashing (3 daemons running), however, I still have runaway memory issues with clustering the workers. It gets further than before which is great but still leaves about 5-10% of blocks unchecked because the heap just keeps getting filled. It's almost certainly a garbage collection issue.

vrfebet commented 5 years ago

can you please describe how to use more demons? Thank you

uaktags commented 5 years ago

I'll be making some documents to share some basic setup ideas. The method that worked for me in a very simple dev setup is to simply clone the VM / DockerContainer you have.

If you can't clone your VM (baremetal server), then simply just spin up a VM. Once you've created a new vm and have the daemon installed, just copy over the blocks/datafiles from your good VM to it and then launch the daemon. Make sure that your <bitcoind>.config file is updated for the proper rpcallowip of your nodejs server.

Now you just have to contend with networking. If there's a LAN for your VMs/Servers, then great HAProxy can be setup on the master server or on a new server, and use that as your loadbalancer for calls. Or you have a hosting provider that already provides you a loadbalancer, like ionos does for their cloudserver, then use that.

uaktags commented 5 years ago

This appears to be working just fine. https://github.com/uaktags/explorer/wiki/Loadbalance-Coin-Daemon-with-HAproxy