CPU and RAM deployment parameters.

[brunodonassolo]

Hi @HaraldGustafsson

Here it is the first implementations to add CPU and RAM parameters as discussed in https://github.com/EricssonResearch/calvin-base/issues/83.

If you could give a look and say me your opinion so I can improve the missing points. It depends on some external tool to update the CPU/RAM available in the runtime during execution and only consider them at the initial deployment.

Regards,

Commit message:

Now it is possible to select the runtimes to run actors based on some CPU and RAM parameters.

CPU parameters:

• Total CPU: power in MIPS of runtime. The acceptable values are: 1 MIPS, 1000 MIPS, 100000 MIPS, 1 GIPS, 10 GIPS
• Available CPU: percentage of CPU available. Acceptable values: 0%, 25%, 50%, 75%, 100%.
• Affinity: runtimes runs in a unique CPU core. Acceptable values: "dedicated".

RAM parameters:

• Total RAM: Total memory available for runtime. Acceptable values: 1Kb, 100Kb, 1Mb, 100Mb, 1Gb, 10Gb
• Available RAM: Percentage of memory available. Acceptable values: 0%, 25%, 50%, 75%, 100%.

Other values must be rounded up/down if necessary.

2 groups of parameters:

• Fixed: they are attributes configured by command line when the runtime is started. It contains: cpuTotal, cpuAffinity and memTotal.
• Variable: they can be changed during execution through the control API. It includes: cpuAvail and memAvail.

How to use:

• Fixed: as another indexed_public parameter: csruntime --name runtime-3 -n 192.168.0.2 --attr '{"indexed_public": {"memTotal": "100K", "cpuTotal": "100000"}}'

In the deployjson file, node has at least 1000 MIPS of CPU power: {"requirements":{"snk":[{"op":"node_attr_match","kwargs":{"index":{"cpuTotal": "1000"}},"type":"+"}],

• Variable: The deployjson file is similar to the fixed one. For at least 25% of CPU available: {"requirements":{"snk":[{"op":"node_attr_match","kwargs":{"index":{"cpuAvail": "25"}},"type":"+"}],

The update is done through control API, e.g. setting 50% of CPU available: curl -X POST -d '{"value": 50 }' http://192.168.0.1:5001/node/resource/cpuAvail curl -X POST -d '{"value": 50 }' http://192.168.0.1:5001/node/resource/memAvail

Internal details:

• Storage structure for: cpuAvail and memAvail /nodeCpuAvail-ID and /nodeMemAvail-ID: save percentage for ID runtime /index-/cpuAvail/0/25/50/75/100 and /index-memAvail/0/25/50/75/100: save the runtimes that have at least the percentage of resource available. So, the ID of runtime-1 that has 50% RAM available can be recovered by searching the lower indexes: /index-/memAvail/0/25 for example.

• Storage structure for: cpuTotal, memTotal and cpuAffinity. Follows the structure of other public indexed attributes.

[HaraldGustafsson]

I will take a look at this tomorrow.

[HaraldGustafsson]

I merged your additions. I will likely rewrite some parts of the ResourceMonitorHelper, since I'm anyway in the process of changing the storage API towards the storage plugins (like DHT). I will expose the index and other more advanced operations directly since some storage plugins will have a better way to handle it. One of the methods I now see should handle updating index, seems like tedious to remove and then add again, also runs the risk of race conditions when not waiting between operations. I'm also developing a SQL-database storage plugin.