wdm0006 / DummyRDD

A pure python mock of pyspark's RDD
http://wdm0006.github.io/DummyRDD/
BSD 3-Clause "New" or "Revised" License
27 stars 13 forks source link

`RDD.mapPartitions` should provide `iteratro` or `list` instead of concrete element #34

Open adrian-dankiv opened 6 years ago

adrian-dankiv commented 6 years ago

RDD.mapPartitions in the mock works like RDD.map but shouldn't.

import dummy_spark
rdd = dummy_spark.RDD([1, 2, 3], None)
_ = rdd.map(print) # should print 1\n2\n3\n
1
2
3
# OK!
_ = rdd.mapPartitions(print) # should print [1, 2, 3] or <iterator object>
1
2
3
# FAIL!