Add monitoring of AWS Cloud

[matthi-g]

• [x] Check if cloud is reachable
• [x] Check latency to cloud regions (e.g. use this
• [x] Use AWS price list api to get current pricings

[stefanpedratscher]

Hi @matthi-g , thank you for creating.

I am currently discussing with @zahranajaf

We wrote in the agreement: To keep track of these resources it is necessary to implement a monitoring service that observes metrics like the online state, utilization (memory, cpu and disc), costs, energy consumption, co2 emissions, latency, bandwidth and others. Based on these metrics and the SLOs that are set by the client, resources are proposed that can then be reserved by the client.

What about these functionalities? Do we get them e.g. for EC2/openFaaS?

[matthi-g]

I think some of these are basically impossible to retrieve and I think we already talked about that in the past. At least co2 emissions and also energy consumption can not be monitored in a feasible way. That would be a whole new project. OpenFaas has a prometheus metrics endpoint but after looking it up it turns out, it's useless for these low level metrics. What we could do instead, install a Prometheus Node Exporter which exposes some of these low level metrics. After a quick google search I found found this: https://linuxhit.com/prometheus-node-exporter-on-raspberry-pi-how-to-install/ I would go with something like this and do the same for ec2.

Let me know what you think @stefanpedratscher

[stefanpedratscher]

Yes, this sounds not that bad. What do you think @zahranajaf ?

[matthi-g]

Monitoring of latency, up status and price list is implemented through a custom solution in vertx. Metrics regarding ec2 resources are exposed with node exporter and scraped by victoria metrics