search

wdm0006 / DummyRDD

A pure python mock of pyspark's RDD
http://wdm0006.github.io/DummyRDD/
BSD 3-Clause "New" or "Revised" License
27 stars 13 forks source link

Partition based bug #25

Closed mrkaiser closed 7 years ago

mrkaiser commented 7 years ago

Spark RDDs pass iterables to foreachPartition unlike foreach (which gets every row of the rdd one at a time). This means we have to pass the full list into foreachPartition in DummyRDD to keep witht spark api

coveralls commented 7 years ago

Coverage Status

Coverage increased (+0.8%) to 81.053% when pulling 88d0ef4981be14a94f62804dc5d98b5ac70fd613 on mrkaiser:hotfix/partition_bug into 691d4c371a1affee7d9cc6ad2b44553ff93c2eb7 on wdm0006:master.

coveralls commented 7 years ago

Coverage Status

Coverage increased (+0.8%) to 81.053% when pulling 88d0ef4981be14a94f62804dc5d98b5ac70fd613 on mrkaiser:hotfix/partition_bug into 691d4c371a1affee7d9cc6ad2b44553ff93c2eb7 on wdm0006:master.

coveralls commented 7 years ago

Coverage Status

Coverage increased (+0.8%) to 81.053% when pulling 88d0ef4981be14a94f62804dc5d98b5ac70fd613 on mrkaiser:hotfix/partition_bug into 691d4c371a1affee7d9cc6ad2b44553ff93c2eb7 on wdm0006:master.