Before the new disk scheduler was integrated into the rest of the Spark code, the CacheManager was responsible for making sure that two tasks did not compute the same block at the same time (thus doing duplicate work). Given that the CacheManager has been removed, we need to reimplement this functionality somewhere else.
Before the new disk scheduler was integrated into the rest of the Spark code, the CacheManager was responsible for making sure that two tasks did not compute the same block at the same time (thus doing duplicate work). Given that the CacheManager has been removed, we need to reimplement this functionality somewhere else.