differentiate between number of CPUs to use in idle and busy mode

[romw]

Reported by KSMarksPsych on 12 Oct 37296831 05:46 UTC Posted by: Date: 3:05 PM 04-07-2007 BOINC detects if the machine is busy and can limit the number of CPUs to use. I'd like to see a feature, that I can define 2 numbers of CPUs to use, one if the machine is idle and one if the machine is busy. Right now I've got a dual processor machine and allow BOINC to use one of them so that my work is not slowed down. But when the machine is idle, BOINC could use both CPUs. Right now, this is not possible. With the advent of multi-processor/core machines spreading out to end users, this would allow more CPU power to be used by BOINC and the necessary preconditions are already implented.

Migrated-From: http://boinc.berkeley.edu/trac/ticket/41

[romw]

Commented by romw on 19 Jan 37319314 17:20 UTC What do you think?

[romw]

Commented by davea on 9 Nov 37319324 11:33 UTC I'm not in favor of this. I don't think limiting the number of CPUs used while busy will increase system responsiveness. Limiting the RAM usage will.

E.g.: if you use 1 CPU and it causes thrashing, your system will be sluggish. If you use N CPUs (at zero priority) with small working set, your system will not be sluggish

[romw]

Commented by Pepo on 23 Sep 38902026 21:46 UTC According to numerous requests, I'd like to suggest this being implemented arbitrarily. Currently the client considers just whether to go or sleep and how much memory to use, according to user's idleness.

Both ''Working'' and ''Idle'' states should contain a separately manageable set of ''"resources usage preferences"'' (CPU % throttling, %/number of CPU cores, memory usage, exclusive_app, etc.) just like it is now with preference sets for venues.

[romw]

Commented by Nicolas on 18 Jun 38902341 14:40 UTC http://boinc.berkeley.edu/dev/forum_thread.php?id=3343

[romw]

Commented by Pepo on 20 Apr 40971453 15:06 UTC This way of managing BOINC preferences gets requested multiple times a year, again and again and again... Thus, let's not forget the proposal from a year ago. And best wishes to the related followup discussions.

[romw]

Commented by Ageless on 9 Jan 41894116 15:33 UTC And the latest request.

[RichardHaselgrove]

And it's coming up again in the community responses to the request for input to the current working group. This is one of the oldest trac tickets still open - it's over ten years old, and probably more relevant to modern multi-core computers than it was then.

[Ageless93]

Same request in #202

[mchelen]

One challenge with this seems to be that there a number of different suggestions:

• Allowing different % of CPUs to be used depending on busy/non-busy
• Allowing different % of CPUs to be used depending on time of day
• Allow control of % GPU usage
• Allow lower CPU task priority
• Overall refactor of prefs & prefs UI

Is that a good summary? http://boinc.berkeley.edu/trac/wiki/PrefsRemodel covers a lot of this and more. Is there any way to break this down into more manageable chunks? It also seems like some features (% GPU, CPU task priority) might require implementation in BOINC before Manager could even use them.

[davidpanderson]

Any significant features would require coordinated changes in a) the client, b) the Manager, and c) the web code.

The idea of PrefsRemodel is to define a framework that's both simple and powerful. Namely, to separate the "triggers" (time of day, computer in use, maybe "app X is running", and so on) from the various parameters being controlled.

[sirzooro]

When you will be working on it, please allow to define multiple profiles. I would like to have following ones:

• Full power: use all CPUs and GPU;
• Idle: no CPU/GPU used;
• Gaming: 75% CPUs, no GPU;
• PrimeGrid challenge: 50% CPUs, GPU.

It also would be nice if Gaming profile could activate automatically when BOINC detects full-screen app. There is separate issue for detecting that: https://github.com/BOINC/boinc/issues/2301

[Overtonesinger]

@sirzooro 👍 +1 for the "PrimeGrid challenge" prefereeeeeeeence! YEEEEEEEEEAH! :-) 🥇

[LinAGKar]

I don't think limiting the number of CPUs used while busy will increase system responsiveness.

At the very least, higher CPU usage causes more noise.

[wiedenhoeft]

I second this request, though I think HPC servers would actually benefit from a more general, greedy model (not sure this would justify opening a new feature request). Say you have 100 cores, and no user is running computations, then BOINC should run on 100 cores (or whatever max% the user sets). If a user starts 10 tasks, BOINC should suspend 10 tasks, but keep running the other 90. I understand that this is somewhat emulated using niceness, but the trouble is that there are company and institution environments where (1) BOINC is run on the condition that it does not interfere the slightest with the business-related computations (even niceness 19 would run some % of CPU time), and (2) users are required to run their task with high niceness so as to maintain stable and smooth SSH and other interactive sessions. Right now, it appears to me that the only option is to set max CPU usage to say 90% and suspend BOINC once 10% user CPU is requested, which is wasteful in terms of the computational resources that BOINC could theoretically utilize.

[cminnoy]

Really would love to see it implemented, and also controllable from boinccmd.

[cminnoy]

There is some relation with this: https://github.com/BOINC/boinc/issues/4181

[springfielddatarecovery]

One benefit of implementing this feature is that you could then run GPU tasks while the computer is in use and only use one of your cores for managing them.

[sinusoid2]

Has there been any movement or thought on this? I have tried looking at the code, but I honestly don't know where to start

[davidpanderson]

I wrote a design for a super-flexible prefs system, but realistically that's not going to happen. I think the best thing is to add a couple of prefs to the existing system, and reshuffle it to make it more logical. I'll look into this.

[sinusoid2]

Where could I find this in the code to look at it myself? I may be naive in thinking this, but couldn't it be a simple if/then or case/switch statement? INUSE ##% cores, EXCLUSIVEAPP ##% cores, ##% cores?

[AenBleidd]

@sinusoid2, that's not just a logic change, there should be also a user settings to control this and set preferred values for idle and busy mode. If you want to contribute to this, I suggest you to start looking at existing settings and their usage to understand to current logic of the scheduler and then try to understand how to modify it

[davidpanderson]

Here's a scheme with a couple of new prefs, organized better. Comments?

When computer is in use "in use" means no input in X min Use at most X% of the CPUs Use at most X% of the CPU time suspend if non-BOINC CPU usage exceeds X% Suspend all computing Suspend GPU computing Use at most X% of memory

When computer is not in use Use at most X% of the CPUs Use at most X% of the CPU tie suspend if non-BOINC CPU usage exceeds X% Use at most X% of memory Suspend (allow computer to sleep) if no input in last X min

Jobs suspend when on battery switch every X checkpoint every X leave non-GPU jobs in memory while suspended use at most X% of page/swap store at least X store up to an additional X compute only between...

Disk use no more than X leave X free use no more than X%

Network limit upload/download rate transfer files only between... limit usage to MB every X days skip verification for image files confirm before connecting disconnect when done

[makeasnek]

David's proposal sounds lovely

[sinusoid2]

David, looks good. Small fix, your first line un in use should be something along the line of "in use" means input in X min (pull out the "no")

[davidpanderson]

I implemented the new prefs in the dpa_niu_prefs branch. The web interface now looks like this: https://setiathome.berkeley.edu/prefs.php

Please review; I want to make sure people like this before making the corresponding changes in the Manager.

[makeasnek]

I implemented the new prefs in the dpa_niu_prefs branch. The web interface now looks like this: https://setiathome.berkeley.edu/prefs.php

Please review; I want to make sure people like this before making the corresponding changes in the Manager.

Looks like one needs a login to see this? I am so excited for this change, this will easily double the amount of computational power I can give to BOINC

[Vulpine05]

It looks like SETI@home is used as a live test, so if you have SETI@home you can log in. If not, here is a screen capture:

@davidpanderson, if we're looking for a broader pool of people to review and comment, I could post this on the BOINC message board (maybe a few other projects as well, too). Let me know what you think.

[makeasnek]

This looks awesome!!

[davidpanderson]

@Vulpine05 Feel free to gather feedback - thanks.

[Vulpine05]

@davidpanderson will do. Its getting late and I want to be sure the message is well written, so I will probably get to it tomorrow. In the meantime, I'm going to play devil's advocate.

On one hand, this looks pretty good. My only comments on this format would be:

1. Suspend when non-BOINC CPU usage is listed in both 'When computer is in use' and 'When computer is not in use'. I think it should only be in the 'When computer is in use' header.
2. I think the "tasks" header should be changed to "General". When I read "Tasks", I think it means for each individual task. Battery suspension and storing work, for example, apply to all the tasks.

On the other hand, I am sure some users are going to miss having CPU, network, and disk/memory settings bundled into those categories. So that everything doesn't have to get re-formatted and moved around, how about adding a second set of CPU usage and CPU time? Here is an example I just made in Paint:

[sinusoid2]

Looks good to me. I would like to see a further step personally. I know there are exclusive apps that stop it all together, but with so many cores even exclusive apps could probably just be limited to a user preference as well.

[davidpanderson]

That sort of thing would be handled by the ultra-general framework described here: https://boinc.berkeley.edu/trac/wiki/Prefs2 However, implementing that would be a gargantuan task.

[davidpanderson]

I'll change "Tasks" to "General".

"non-BOINC CPU usage" appears in both places because there are potentially 2 different limits, e.g. a low limit while in use, and a higher limit while not in use (but possibly still running CPU-intensive jobs, like a server node).

[Vulpine05]

@davidpanderson, what changes, if any, will be in simple view?

I think we should move the conversation for reformatting the preferences to another issue, also.