search

amplab / graphx

Former GraphX development repository. GraphX has been merged into Apache Spark; please submit pull requests there.
https://github.com/apache/spark
Apache License 2.0
360 stars 103 forks source link

Load edges in columnar format #94

Closed ankurdave closed 10 years ago

ankurdave commented 10 years ago

Fixes indentation. Diff ignoring whitespace: https://github.com/amplab/graphx/pull/94/files?w=1

In GraphLoader.edgeListFile, load edges directly into EdgePartitions, avoiding repartitioning. Instead, allow repartitioning by calling Graph.partitionBy(partitionStrategy).

AmplabJenkins commented 10 years ago

Merged build triggered.

AmplabJenkins commented 10 years ago

Merged build started.

AmplabJenkins commented 10 years ago

Merged build finished.

AmplabJenkins commented 10 years ago

One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/2620/

jegonzal commented 10 years ago

This is good!  Do we also have a way to repartition?  Perhaps in general the graph should not shuffle edges unless it is requested. This allows users to do their own problem specific partitioning.

Actually you might load your data into GraphX then map triplets to get some information about both vertices (eg where they live) and then attach an edge attribute (eg common zip code) and finally rebuild the graph by first externally shuffling the edges on the zip code. 

— Sent from iPhone

On Sat, Dec 7, 2013 at 6:25 AM, UCB AMPLab notifications@github.com wrote:

One or more automated tests failed

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/2620/

Reply to this email directly or view it on GitHub: https://github.com/amplab/graphx/pull/94#issuecomment-30055888

ankurdave commented 10 years ago

I like the idea of never shuffling edges unless requested. Maybe we could expose a method Graph.partitionBy(PartitionStrategy) and let the user call it?

AmplabJenkins commented 10 years ago

Build triggered.

AmplabJenkins commented 10 years ago

Build started.

AmplabJenkins commented 10 years ago

Build finished.

AmplabJenkins commented 10 years ago

One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/2708/

AmplabJenkins commented 10 years ago

Merged build triggered.

AmplabJenkins commented 10 years ago

Merged build started.

ankurdave commented 10 years ago

This is ready to merge.

AmplabJenkins commented 10 years ago

Merged build triggered.

AmplabJenkins commented 10 years ago

Merged build finished.

AmplabJenkins commented 10 years ago

All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/4088/

AmplabJenkins commented 10 years ago

Merged build started.

AmplabJenkins commented 10 years ago

Merged build finished.

AmplabJenkins commented 10 years ago

All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/4090/

ankurdave commented 10 years ago

Rebased onto the new master.

AmplabJenkins commented 10 years ago

Merged build triggered.

AmplabJenkins commented 10 years ago

Merged build started.

AmplabJenkins commented 10 years ago

Merged build finished.

AmplabJenkins commented 10 years ago

All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/GraphXPullRequestBuilder/4586/

jegonzal commented 10 years ago

Looks good to me.