CCI-MOC / xdmod-cntr

A project to prototype the use of XDMOD with OpenStack and OpenShift on the MOC
1 stars 5 forks source link

define a kubernetes cron job for openstack-hypervisors #115

Open rob-baron opened 1 year ago

rob-baron commented 1 year ago

By fetching the hypervisor information, this will enable the percent utilization statistics within xdmod.

rob-baron commented 1 year ago

The "hypervisor_facts.py" script needs to be update. currently it is producing output like:

{
  "hypervisors": [
    {
      "hypervisor_hostname": "nerc-hyp-0.rc.fas.harvard.edu",
      "id": "6adfb258-d4aa-4d8f-a9da-72ce55375fd9",
      "memory_mb": 386375,
      "vcpus": 96
    },
    {
      "hypervisor_hostname": "nerc-hyp-1.rc.fas.harvard.edu",
      "id": "0954d25d-f577-4fbb-9939-bf8996f5d13e",
      "memory_mb": 386375,
      "vcpus": 96
    },
   ...
    {
      "hypervisor_hostname": "nerc-hyp-25.rc.fas.harvard.edu",
      "id": "8b8b6961-454b-4db2-92f3-320bce7d9793",
      "memory_mb": 386379,
      "vcpus": 48
    }
  ],
  "ts": "2023-04-28T14:36:28Z"
}

But it should be:

{
  "hypervisors": [
    {
      "hypervisor_hostname": "nerc-hyp-0.rc.fas.harvard.edu",
      "id": 1,
      "memory_mb": 386375,
      "vcpus": 96
    },
    {
      "hypervisor_hostname": "nerc-hyp-1.rc.fas.harvard.edu",
      "id": 2,
      "memory_mb": 386375,
      "vcpus": 96
    },
   ...
    {
      "hypervisor_hostname": "nerc-hyp-25.rc.fas.harvard.edu",
      "id": 21,
      "memory_mb": 386379,
      "vcpus": 48
    }
  ],
  "ts": "2023-04-28T14:36:28Z"
}

A cron job should be created similar to:

https://github.com/CCI-MOC/xdmod-cntr/blob/main/k8s/kube-base/cj-xdmod-openstack-shred.yaml

It'll need it's own PV to store the hypervisor json history, so the script to run this will need to run the following:

    xdmod-openstack-hypervisor --cloud nerc-openstack
    xdmod-shredder -r nerc_openstack -d /root/xdmod_data/hv_facts -f cloudresourcespecs
    xdmod-ingestor  --datatype=cloudresourcespecs

see (https://open.xdmod.org/10.0/cloud.html)