search

thinkaurelius / faunus

Graph Analytics Engine
http://faunus.thinkaurelius.com
Apache License 2.0
261 stars 58 forks source link

Make BlueprintsGraphReader/Writer and Input/OutputFormats. #16

Closed okram closed 12 years ago

okram commented 12 years ago

Would be serial, on the initial load, but once in Hadoop, all operations are parallel.

okram commented 12 years ago

This should actually be via Rexster as Hadoop will need to pull this over the wire. Therefore a RexsterReader/Writer and RexsterInput/OutputFormat.

spmallette commented 12 years ago

Would be cool if this could tie into a revised super speedy RexPro.

okram commented 12 years ago

https://github.com/thinkaurelius/faunus/tree/master/src/main/java/com/thinkaurelius/faunus/formats

okram commented 12 years ago

Via RexPro, use the binary representation of a FaunusVertex to send the binary representation over the wire as a Hadoop DataInput.

@spmallette recommended that we make this a Kibble. What is cool is that if we do this, then you simply drop the faunus.jar in your Rexster /ext and BAM, you can start sucking your graph.getVertices() iterator across the wire in Faunus-ready binary format.

okram commented 12 years ago

Done. Thanks @spmallette