TomoMaster Crawler should internally connect to the node

[thanhson1085]

https://github.com/tomochain/infrastructure/blob/f2d091b0e839d2a6641afd2545e0301f5772dfd1/networks/devnet/main/04tomomaster.yml#L26

It could be easy if they are running in a same swarm.

Also take a look for TomoScan crawlers

[etienne-napoleone]

So we have two choice:

• load balanced external access
• single node swarm access

First one have load balancing and redundancy, second one have a bit less latency.

[thanhson1085]

I am thinking about this idea. We have external loadbalances (cloudflare, cloud provider aws, DO). We will also setup a internal load-balance (nginx upstream). So the follow will be: the crawler -> internal loadbalance -> nodes

[etienne-napoleone]

yep can do!

[etienne-napoleone]

implemented in #94