search

madecoste / swarming

Automatically exported from code.google.com/p/swarming
Apache License 2.0
0 stars 1 forks source link

Rip out sharding support #103

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
The fact Swarming support intrinsic sharding is something that has long been 
seen as a core feature that has no equivalent in other task distribution 
mechanism out there.

The main issue with this approach is that it clearly complexifies the server 
code and slows it down. It makes statistics harder to figure out and the task 
requests now becomes a complex mix of requests and shards. This forces the 
server to use more DB transactions than it would otherwise be necessary, 
opening the door to DB inconsistencies. A side effect of this support is the 
implicit baked google-test assumptions.

The best reimplementation is to do it client side directly inside swarming.py. 
swarming.py has all the necessary knowledge to do it and is the right 
abstraction level to do it. It is filed as a separate bug.

AI:
- The client must stop sending sharded requests before the server stops 
supporting them.
- Rip out server side sharding support and simplify the DB accordingly, related 
to bug 89.

Original issue reported on code.google.com by maruel@chromium.org on 8 May 2014 at 7:05

GoogleCodeExporter commented 9 years ago

Original comment by maruel@chromium.org on 29 May 2014 at 1:53