Create optimized base container for our containers with blockchain systems

cyberFund / cybernode-archive

🚀 Manager of docker images for cybernomics

MIT License

19 stars 4 forks source link

Create optimized base container for our containers with blockchain systems #35

Open cyborgshead opened 7 years ago

cyborgshead commented 7 years ago

@abitrolly research this please https://github.com/cusspvz/autotune.docker

cyborgshead commented 7 years ago

@abitrolly check this http://phusion.github.io/baseimage-docker/

abitrolly commented 7 years ago

Let me clarify - this setup is for full blockchain nodes?

I need to get more info about what type of workload to expect from these nodes? Optimizing network stack for 300.000 concurrent connections may be not as worthy as more CPU cycles to query blockchain itself.

autotune looks reasonable. Let's setup monitoring first to check that those IPv4 settings really matter.

abitrolly commented 7 years ago

I am checking what are the best practices to keep containers up to date with security updates in base images - https://stackoverflow.com/questions/26423515/how-to-automatically-update-your-docker-containers-if-base-images-are-updated

And so we also need to decide what would be the base image. 16.04 LTS or 17.04 or something more exotic? I am comfortable with 17.04 though.

cyborgshead commented 7 years ago

@abitrolly LTS version, and yes, full blockchain nodes, but directories with chain data should be mounted

abitrolly commented 7 years ago

Where are those storage mounts physically and logically located?

abitrolly commented 7 years ago

Meeting notes. Blockchain data is mounted to /data point inside container and physically located on the host that runs the full node container for now.

abitrolly commented 7 years ago

Need monitoring to add current nodes to it to gather requirements. Variants:

Digital Ocean Monitoring

PROS: Easy to install. Graphs are inside DO control panel, easily accessible by everyone with login. CONS: Participation in their data collection policy is not an opt-in. Not clear if the monitoring will work with hardware deployed nodes. https://www.digitalocean.com/community/tutorials/an-introduction-to-digitalocean-monitoring

abitrolly commented 7 years ago

Metrics that is needed right now:

[ ] bandwidth usage by month - download/upload rate - peaks and their characteristics
[ ] traffic usage by month - download/upload in GB
[ ] most CPU/memory hungry process/job by month
[ ] disk io bottlenecks and close to bottleneck conditions
[ ] API/service request waiting time and characteristics of those requests

abitrolly commented 7 years ago

Metrics specific to blockchain workload:

[ ] size of blockchain database on disk
[ ] size of database indexes in memory