CPU 0 should be used

[akerbos]

Currently, the cpu with id 0 is not used with the rationale being that this one is often reserved for OS processes.

However, there is a basic scenario for which this assumption is void, namely if the whole Java process is restricted to some otherweise unused cores. Java will only see those it can use, so the library prevents me from using a perfectly usable CPU.

The easiest solution is, of course, to just use CPU 0. It is the last one considered, anyway, so I think this should not cause problems in most situations.

[peter-lawrey]

You can't bind to cpu 0, however without this binding it is left to the OS to schedule the thread, and it can use cpu 0 if it decides it is free.

If you have bound every cpu to Java, what do you expect interrupts and the rest of the system to use?

The assumption is that interrupts and other processes/threads (including other threads in your JVM) are unavoidable and have to run somewhere.

[akerbos]

The scenario is: I have a machine with 24 cores. Even if there is a reserved CPU, I can comfortably assign 6 or 12 cores to my JVM (which won't see that there are more than that physically). In this case, there is no reason not to use CPU 0 (which is 5 or 11 physically).

[peter-lawrey]

This is true, but you can't comfortably bind 24 cpus to your JVM. There has to be at least one cpu, which cannot be bound.

I have added the following to the How it Works page.

CPU Layout

A machine has one or more Sockets (whole chips). Many machine have only one socket, but some have two and a few have four.

Each socket can have a number of cores, between one and eight is common.

Each core can have many threads one or two is common but some have up to 32.

It is these logical threads which appear as individual cpus to the operating system and it is these you can bind to.

[akerbos]

You start counting downwards from Runtime.getRuntime().availableProcessors(). This figure defintely depends on which CPUs are assigned to the JVM, not how many there really are. So you will only try to assign locks for CPUs the JVM can see, which may or may not include the system critical CPU. Is there a reason to leave one CPU free even in the first case?

The explanation is helpful for people like me one week ago who don't know this stuff.

[peter-lawrey]

The idea is isolated a number of cpus for reserving. This means that no program including JVM will try to uses those cpus unless explicitly assigned. This is deliberate because you want to assigned cpus only those threads (not other in the JVM or other processes will use) i.e. you will be using the cpus the JVM cannot by default use.

[akerbos]

Are you saying that your library circumvents core affinity settings the OS made? If so, that is not good. The way I see it, the library should bind/block only such cores that the OS assigned to the current JVM.

In any case, you should check carefully wether Runtime.getRuntime().availableProcessors() really is what you want to use.

[peter-lawrey]

As show in the tests, you get the best results assigning thread which will not be used from any program normally (including the JVM) As such you want to use cpus, thread in the JVM won't normally use either. i.e. you want to opposite to what you suggest.

In this case, you want all the possible available cpu.

[akerbos]

It might speed up my Java application, but at what price? You essentially skrew the whole system! Think of a server which is setup so that different (server) processes run on different cores. If you use your library in this setting as it is, the JVM gobbles up the whole system, circumventing the admin's setup and slowing down -- or even stopping -- the other servers.

I don't think this is a good idea at all. At the very least, there should be a switch to turn this behaviour off and a more environment-aware one on.

[peter-lawrey]

You are right that the cost of using this system is that you are dedicating the server to running particular programs. Its not ideal for adhoc running of programs or servers without some portion of planned work load.

If you want to use multiple programs/JVMs on the same server you need to use a command line argument to specify which cpus can be dedicated to this application.

-Daffinity.reserved=CC

This specifies which cpus can be dedicated to this application. (in hex) Ideally, they should not be cpus which any other application can use (though you can still see an improvement even if they are)

If you using hyperthreading, you could be sharing cores between cpus so you can still use every core even when you are only using half the threads. i.e. you don't need to have 100% utilisation.

[akerbos]

Which brings us back to square one: either you use all cores without considering the external settings. Or you use only those cores externally allowed but prevent the CPU with ID 0 from being used, which is in general undesirable.

[peter-lawrey]

The default behaviour is to allow the OS to schedule thread which don't use this library (which is normal behaviour for a program), and exclusively assign cpus which are not for general use. Its only these cpus which can be assigned exclusively.

If you isolate cpus 2,3,6,7 for example, the base affinity for the JVM will be 0,1,4,5 and threads which are not assigned will use those cpus (determined by the OS) The library will attempt to assign cpus which can be assigned exclusively as its this exclusivity which yields the best performance advantage. If you don't want a process to use all the isolated cpus (or you want to be able to assign non-isolated cpus), you can specify on the command line which cpus it can assign.