Add optional dimension filtering in task request, e.g. allow a dimension to be a list instead of a single value

[GoogleCodeExporter]

Goal:
Guarantee a minimum throughput for each consumer teams/project by assigning 
dedicated pools to subteams by dedicating bots.

Use case:
Let's say there's dedicated pool of 10 devices for team A, another pool of 10 
for team B, and a 40 shared pool usable by both team A, B and C. Teams A and B 
are guaranteed basic throughput and can each expand on the shared pool of 40 
bots. Team C is "free riding" the shared pool. This is orthogonal to task 
priority, so that each team can stomp on each other with higher priority task 
as needed.

The way to specify that is that you state an optional dimension:
  ./swarming.py trigger ... -d lab sharedpool -d lab teamApool

It would allow a device assigned to the team A dedicated pool or one in the 
shared tool, whatever is available first. Note that priorities are still taken 
in account, it just enables more seamless sharing of different pools to get 
dedicated throughput.

Implementation:
https://code.google.com/p/swarming/source/browse/services/swarming/server/task_t
o_run.py#45
TaskToRun is the key hash. What we'd want is the hash to be a list. The way to 
do this is to add a new property:
hashes = ndb.IntegerProperty(repeated=True, indexed=False)
so that the loop at 
https://code.google.com/p/swarming/source/browse/services/swarming/server/task_t
o_run.py#345
would do instead:
if not any(h in accepted_dimensions_hash for h in task_key.hashes):

The change can be done live by enabling the store of the new property in 
TaskToRun first, waiting for all pending tasks since the instance switch over 
to be done, then switching yield_next_available_task_to_dispatch() to use the 
new logic and stop using the entity key.

Non-use case:
[This is already supported and listed for demonstrative purposes]
A select number of bots have a specific configurations. Let's say they are 
equipped for performance tests that require a sensor to be associated with it. 
The layout would be as follow:
perf bots would have dimension depending on their physical setup:
- {'temperature_sensor': '1', 'camera': '0'}
- {'temperature_sensor': '0', 'camera': '1'} 
- {'temperature_sensor': '1', ''camera': '1'}
normal bots would have dimension {'temperature_sensor': '0', 'camera': '0'}

For use case where a camera recording the screen is needed, the request could 
be:
{'camera': '1'}
so that bots with or without 'temperature_sensor': '1' would be used.

But bots with a temperature sensor are rare and will likely be starved! The 
right way to handle this use case is to use higher priority for tasks that 
requires limited throughput bots. This is why closely tuning priorities is 
important.

Original issue reported on code.google.com by maruel@chromium.org on 15 Aug 2014 at 4:04