issue on parallel ingress using obvious heuristics

jegonzal / PowerGraph

PowerGraph: A framework for large-scale machine learning and graph computation.

340 stars 517 forks source link

issue on parallel ingress using obvious heuristics #18

Open weristdas opened 11 years ago

weristdas commented 11 years ago

It seems that thers is no general thread-level parallel support for this:

file: ./graphlab-master/src/graphlab/graph/distributed_graph.hpp 1902 #ifdef _OPENMP 1903 #pragma omp parallel for 1904 #endif 1905 for(size_t i = 0; i < graph_files.size(); ++i) { 1906 if ((parallel_ingress && (i % rpc.numprocs() == rpc.procid()))

When loading from multiple files by multiple processes on 'obvious mode', openmp doesn't work correctly here.

ylow commented 11 years ago

Hmm... Obvious mode?

weristdas commented 11 years ago

yes, e.g. run pr like this: mpirun -n 16 --hostfile ~/slots ~/bin/pagerank --graph_opts="ingress=oblivious" --graph /mnt/nfs/twitter/ --format tsv

在2013-08-14 01:46:53，闫洁yanjie@ict.ac.cn写道：

Hmm... Obvious mode?

— Reply to this email directly or view it on GitHub.

ylow commented 11 years ago

We need a bit more details on this issue. How many files are in the directory /mnt/nfs/twitter ? And in what way does it fail?

ylow commented 11 years ago

Can you provide some output logs?