search

LuxGraph / Lux

A Distributed Multi-GPU System for Fast Graph Processing
Apache License 2.0
63 stars 8 forks source link

Support of data structs binding on V/E layer #1

Open archenroot opened 5 years ago

archenroot commented 5 years ago

Hello,

this work looks superior, going to run some stuff on and test it. Especially against GUNROCK project.

Q: Do you think about supporting binding data structs to V/E, which can be used to limit/restrict traversal algorithms implemented?

jiazhihao commented 5 years ago

Thanks for your interests in Lux!

I am not sure if I fully understand what do you mean by binding data structs to V/E. Currently we use CSC/CSR format to store graphs internally, and both node and edge properties can be any type (e.g., scalar, vector, etc..)

archenroot commented 5 years ago

@jiazhihao - thanks for prompt answer. I actually didn't yet compile and looked in more detail in API, but I already looked at Legion and other GASNe yesterday :-) very nice stuff.

I discussed similar topic with guys from GUNROCK team: https://github.com/gunrock/gunrock/issues/453

Also guys from GUNROCK actively participate in newly released project by NVIDIA and its partners RAPIDS platform which resulted in sub-project: https://github.com/rapidsai/cugraph

.... these are just information for you if you are not aware yet :-)

But back to my question (from gunrock issue):

Scenario: Simple graph of 4 nodes (square) with properties (Strings, can be in form of vector of course): Vertex id: A (number, but for clear representation)- property: alfa | Edge id: AB - property: USE THIS PATH Vertex id: B - property: LAMBDA | Edge id: BC - property: USE THIS PATH Vertex id: C - property: PHI | Edge id: CD - property: DONT USE PATH Vertex id: D - property: OMEGA | Edge id: DA - property: DONT USE PATH

Now I would like to search in such structure for SHORTEST PATH trough edges where property=USE THIS PATH.

Can I achieve such effect with Lux at moment? This is to to demonstrate real world benchmark.

Also more details on similar discussion you can find at referenced issue from GUNROCK project, especially I discuss the idea of Mega-KV project integration potential for efficient large data binding to N/E in graph structures on GPU.

jiazhihao commented 5 years ago

@archenroot Thanks for the references.

Yes, Lux can support your scenario. If the edge properties are USE or DONT USE, a simple solution would be treating the edge value as 1 and infinite in the SSSP example. You can also define any complex data structure along with edges/nodes.

We also have a collaborative filtering example whose nodes' properties are vector representations.