search

GGiecold-zz / Cluster_Ensembles

A package for combining multiple partitions into a consolidated clustering. The combinatorial optimization problem of obtaining such a consensus clustering is reformulated in terms of approximation algorithms for graph or hyper-graph partitioning.
MIT License
69 stars 43 forks source link

Parellel processing #6

Closed Jack-Lin-DS-AI closed 7 years ago

Jack-Lin-DS-AI commented 7 years ago

Does current version allow python to call more than twice Cluster_Ensembles on the same machine? Thank you.

Jack-Lin-DS-AI commented 7 years ago

I tried to create and enter a temporary folder for each time I execute Cluster_Ensembles, but it didn't work.

GGiecold-zz commented 7 years ago

I do not see any obstacle to processing in parallel (with the multiprocessing or the concurrent.futures modules from the Python standard library) multiple, distinct ensembles of partitions, provided each Cluster_Ensemble job is called with its own dedicated *.h5 file.

As for a multiprocessing implementation of a single consensus clustering task, that would involve the kind of complications that have been addressed in Concurrent_AP (lock conditions on an HDF5 file, etc.).

Gregory

On 10/24/16 5:55 PM, Chih-Hsu Lin wrote:

Does current version allow python to call more than twice Cluster_Ensembles on the same machine? Thank you.

GGiecold-zz commented 7 years ago

Willing to share any detail?

On Oct 25, 2016 1:54 PM, "Chih-Hsu Lin" notifications@github.com wrote:

I tried to create and enter a temporary folder for each time I execute Cluster_Ensembles, but it didn't work.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/GGiecold/Cluster_Ensembles/issues/6#issuecomment-256110158, or mute the thread https://github.com/notifications/unsubscribe-auth/AK3j5-05qXynBPTcAvN7hlt7xnLdYbPeks5q3kI9gaJpZM4KfVKg .

Jack-Lin-DS-AI commented 7 years ago

I found I did something wrong in my code. And it works now. Thank you.