Open loftusa opened 3 years ago
Sureee
@rajpratyush if you're up for it, it'd be great to see some figures comparing runtime in different situations with+without parallelization in the PR as well. Could be fun as well :)
Sure let me try it
I was going through the documentation of Joblib
right now. and I think this should be the proper replacement for the code that you had mentioned.
from joblib import Parallel, delayed
embeddings = [ Parallel(n_jobs=2)(delayed(selectSVD( graph, n_components=n_components, algorithm=self.algorithm, n_iter=self.n_iter, ) for graph in graphs) ]
I was going through the documentation of
Joblib
right now. and I think this should be the proper replacement for the code that you had mentioned.from joblib import Parallel, delayed
embeddings = [ Parallel(n_jobs=2)(delayed(selectSVD( graph, n_components=n_components, algorithm=self.algorithm, n_iter=self.n_iter, ) for graph in graphs) ]
Yeah, that looks right! See this older code in the case branch for another example
So i will push the PR tommorrow morning
in the initial embedding step (ASE on all the graphs separately) prior to concatenation, MASE just does all the embeddings in a for-loop. This can be parallelized pretty easily with
joblib
. I can make a PR, but might also be fun for someone else to play around with (@rajpratyush?). Should be pretty straightforward, you can just parallelize the loop right?this line of code