search

mesos / hadoop

Hadoop on Mesos
176 stars 80 forks source link

Map/Reduce slot allocation is not ideal for small clusters #46

Open tarnfeld opened 9 years ago

tarnfeld commented 9 years ago

In some situations, when using this framework in a very small mesos cluster (enough for only a few tasks) only map slots will be allocated and zero reduce slots. This can cause a dead-lock and although the effect should be reduced by the introduction of #33 (once the map slots became idle, reduce slots would take oveR) in some cases with certain resource allocations it can still occur.

I guess we should do a better job at allocating a decent map/reduce slot ratio.

tarnfeld commented 9 years ago

The ticket #47 is also a little relevant here.