Open adrian-dankiv opened 6 years ago
RDD.mapPartitions in the mock works like RDD.map but shouldn't.
RDD.mapPartitions
RDD.map
import dummy_spark rdd = dummy_spark.RDD([1, 2, 3], None) _ = rdd.map(print) # should print 1\n2\n3\n 1 2 3 # OK! _ = rdd.mapPartitions(print) # should print [1, 2, 3] or <iterator object> 1 2 3 # FAIL!
RDD.mapPartitions
in the mock works likeRDD.map
but shouldn't.