The objective of this issue is to use Prometheus and Prometheus node-exporter to obtain CPU consumption and other information from the hosts, this will be done with the aim of creating a mixed YANG model between Prometheus obtained parameters and Netflow.
The final output will be having two models, one for Netflow data and another for Prometheus data, and be able to mix them with a Apache Flink Aggregator App.
It exist a cryptomining scenario that uses Ansible to run some cryptominers, this hosts have installed a softflowd, taking advantaged of this, it will be easier to install Prometheus node-exporters in all of them to combine Netflow with Prometheus information. The hosts are accessible via OpenStack in the Spider Project with image name spider-crypto
[x] Create a template for Prometheus and deploy in kubernetes
[x] Install a Node-Exporter in a spider-crypto instance
[x] Test that Node-Exporter and Prometheus are communicating correctly
[ ] Create a flink application that scrape prometheus and write results in a kafka topic (HW_crypto)
[ ] Flink application publish received Netflow from the same machines in another kafka topic (Netflow_crypto)
[ ] A Flink aggregation app is created to read from HW_info and Netflow_crypto and merge and aggregate the data
idomingu
commented
1 year ago
DESCRIPTION
The objective of this issue is to use Prometheus and Prometheus node-exporter to obtain CPU consumption and other information from the hosts, this will be done with the aim of creating a mixed YANG model between Prometheus obtained parameters and Netflow. The final output will be having two models, one for Netflow data and another for Prometheus data, and be able to mix them with a Apache Flink Aggregator App.
It exist a cryptomining scenario that uses Ansible to run some cryptominers, this hosts have installed a softflowd, taking advantaged of this, it will be easier to install Prometheus node-exporters in all of them to combine Netflow with Prometheus information. The hosts are accessible via OpenStack in the Spider Project with image name spider-crypto