If you just have "high" and "low", devs will abuse it (mark everything as high)
In practice, there are a limited number of high performance threads available
We can't necessarily share the number of high performance threads with the dev (fingerprinting risk, maps to number of P-cores, reveals details of platform, etc).
Idea: Use forced ranking for "high" workers:
Within Workers marked as "high", devs need to place them in order from most to least important
This can be done with a number, e.g. 1, 2, 3 - with no duplicates, lowest number most important
Workers would be assigned to the "high" thread pool in order by rank; when the high thread pool is exhausted, the remainder will be run on the "low" pool
A default rank can be assigned using the order of Worker creation, but an explicit rank can also be provided to avoid having to re-juggle code order when deciding on priority
No rank needed or implied for "low" Workers
Default value should be "low" to encourage use of E-cores and improve energy efficiency in default case
Problems:
Idea: Use forced ranking for "high" workers: