search

radical-cybertools / radical.analytics

Analytics for RADICAL-Cybertools
Other
1 stars 1 forks source link

Fix/issue 73 #74

Closed iparask closed 5 years ago

iparask commented 6 years ago

This branch fixes the utilization method. For every consumer entity it gets the resource mapping and counts the resources that consumer consumed.

In addition, I added a check whether the resource is cores or gpus. An error is raised when we try to find the utilization of a resource that does not exist in the mapping.

andre-merzky commented 6 years ago

Sorry for the delay on this. I am hesitant to embrace this approach. Without a doubt it is correct, but it ties analytics to an internal data structure of RP which it should have no knowledge of.

I can only think of one 'correct' approach (for some definition of correct): in rp.utils, when getting the RA session info, evaluate the unit and pilot's resource consumptions and perform a similar calculation, then add the result as something like

resources = {'cores' : 10, 'gpus' : 3}

One could make this fully generic by using something like

resources = {'cores' : [number, t_start, t_stop], 
                     'gpus' : [number, t_start, t_stop]}

the RA wouldn't even need to know during what states / events those resources have been used. Positive numbers can be used for available resources (cores in a pilot), negative numbers for resources used (cores for a unit).

What do you think - overkill? But its getting closer to the notion of metrics MT would ultimately like to see in RCT & analytics...

iparask commented 5 years ago

Let's do it this way, yes. I am not sure why though we forgot about it. Do you want to take over on this one?

andre-merzky commented 5 years ago

Yeah, I don't mind doing that - but probably not right now... :/

andre-merzky commented 5 years ago

I am going to merge this now, and move the request to clean up to PR #88