new pscheduler 'availability' command

[bltierney]

It would be very useful to have pscheduler command to query roughly how busy a host is? something like ‘pscheduler availability —host=hostname’ that would return a %percent remaining free time for tests that require 'exclusive' access.

This could then be used by various GUIs to indicate how busy the host is before adding it to a test mesh.

[marian-babik]

+1 for WLCG, it would great to have this available via API, so we're able to tune up/down our central meshes based on how busy the hosts are. We now rely on calculating this manually, which works fine if all tests are centrally defined (via meshconfig), but doesn't if sites add (significant number of) their own custom tests locally.

[mfeit-internet2]

@marian-babik All operations involving pScheduler's timeline are done through its REST API, so what you want will be available by pulling the schedule and analyzing it. For the moment, the API isn't formally documented, and that's something I hope to get to after 4.0 is out the door.

There's a lot of power hidden by meshconfig and its limitations, but we didn't have the horsepower to do a major overhaul in this release and opted instead to leave the front side of it the same and have it talk to pScheduler from the back. (We may consider a complete replacement for a future release.)

pScheduler is reasonably intelligent, and the best thing to do is give it enough leeway to do its work. I recommend the following:

• Set the start time and repeat interval to reflect the ideal times for the measurement to be taken (e.g,. start at 2017-02-20T00:00:00 and repeat every PT1H for measurements at the top of every hour).
• Allow as much slip as is tolerable (e.g., PT30M for up to 30 minutes of slip from the ideal, top-of-the-hour start time). More slip increases the probability that there will be a time available to run the measurement. pScheduler will find the time if it's there.
• Make the amount of slip random using the sliprand switch. This will randomize the time selected within the available slip and helps prevent runs from getting bunched up at the top of the hour, making ad hoc testing more difficult.

[mfeit-internet2]

After discussion:

• [ ] Availability for exclusive tasks over some time period. "x% free for exclusive over the next hour"