I'm using an extra function to create a new scope for the id variable in each iteration. See #4 for more details.
Another potential fix is to include id as a default parameter in the lambda function like
data = [rdd.map(lambda x, id=id: (x, id)) for id, rdd in data.items()]
but that seems more hacky.
This pull request also includes some changes to .gitignore because I wasn't sure how to exclude them :frowning_face:
I'm using an extra function to create a new scope for the
idvariable in each iteration. See #4 for more details.Another potential fix is to include
idas a default parameter in the lambda function likedata = [rdd.map(lambda x, id=id: (x, id)) for id, rdd in data.items()]but that seems more hacky.This pull request also includes some changes to .gitignore because I wasn't sure how to exclude them :frowning_face: