amplab / graphx

Former GraphX development repository. GraphX has been merged into Apache Spark; please submit pull requests there.
https://github.com/apache/spark
Apache License 2.0
357 stars 103 forks source link

Changing the API of mrTriplets? #101

Closed rxin closed 10 years ago

rxin commented 10 years ago

I've been thinking about this, and I think the mapUDF part of mrTriplets is a little bit confusing because we are asking the user to return an Iterator of (Vid, A).

How about we change mapUDF's signature from EdgeTriplet to (EdgeTriplet, Emitter), and the Emitter has a function sendMessage(targetVid: Vid, message: A)?

This way it should be more efficient, but also more obvious what the mapUDF is doing.

@ankurdave @jegonzal @dcrankshaw

ankurdave commented 10 years ago

That would probably be clearer, though it's a shame to sacrifice immutability/FP principles. If we do this, why not just put sendMessage on EdgeTriplet?

rxin commented 10 years ago

That's a good idea (putting it in EdgeTriplet).

It does make it less "functional". But I don't think it sacrifices immutability though.

ankurdave commented 10 years ago

I worked on this a bit here: https://github.com/ankurdave/graphx/tree/mrTriplets-sendMessage

Compare view: https://github.com/ankurdave/graphx/commit/758e8611d1f8c5ef5debb8034c78053d1e2b9b67