Error running example

[oguerrer]

Hello,

Thanks for your contribution, I am trying to run it in python 2.7, anaconda distribution, following your example and I get the following error when running the cluster_ensembles command:

Traceback (most recent call last): File "", line 1, in File "/Users/me/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/Cluster_Ensembles.py", line 297, in cluster_ensembles store_hypergraph_adjacency(hypergraph_adjacency, hdf5_file_name) File "/Users/me/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/Cluster_Ensembles.py", line 190, in store_hypergraph_adjacency FILTERS = get_compression_filter(byte_counts) File "/Users/me/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/Cluster_Ensembles.py", line 138, in get_compression_filter if 2 * byte_counts > 1000 * memory()['free']: File "/Users/me/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/Cluster_Ensembles.py", line 63, in memory with open('/proc/meminfo') as file: IOError: [Errno 2] No such file or directory: '/proc/meminfo'

Do you have any idea what could be the problem?

Thanks

[GGiecold-zz]

/proc/meminfo is the path to a file in Linux, holding information about free memory.

Cluster_Ensembles was never meant to be compatible with Windows or OS X.

A fix would involve using the psutil module to change the getChunkSize from the Cluster_Ensembles package:

import psutil mem = psutil.virtual_memory() free = mem['free']

Gregory

On Oct 16, 2016 11:57 AM, "Omar Guerrero" notifications@github.com wrote:

Hello,

Thanks for your contribution, I am trying to run it in python 2.7, anaconda distribution, following your example and I get the following error when running the cluster_ensembles command:

Traceback (most recent call last): File "", line 1, in File "/Users/me/anaconda/lib/python2.7/site-packages/ Cluster_Ensembles/Cluster_Ensembles.py", line 297, in cluster_ensembles store_hypergraph_adjacency(hypergraph_adjacency, hdf5_file_name) File "/Users/me/anaconda/lib/python2.7/site-packages/ Cluster_Ensembles/Cluster_Ensembles.py", line 190, in store_hypergraph_adjacency FILTERS = get_compression_filter(byte_counts) File "/Users/me/anaconda/lib/python2.7/site-packages/ Cluster_Ensembles/Cluster_Ensembles.py", line 138, in get_compression_filter if 2 * byte_counts > 1000 * memory()['free']: File "/Users/me/anaconda/lib/python2.7/site-packages/ Cluster_Ensembles/Cluster_Ensembles.py", line 63, in memory with open('/proc/meminfo') as file: IOError: [Errno 2] No such file or directory: '/proc/meminfo'

Do you have any idea what could be the problem?

Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/GGiecold/Cluster_Ensembles/issues/5, or mute the thread https://github.com/notifications/unsubscribe-auth/AK3j51SWrg_fjVkmez2dRzLEtgLJX5Mfks5q0kl7gaJpZM4KYAWm .

[oguerrer]

Thanks a lot. This got me one step closer. Now I am getting the following error:

Traceback (most recent call last): File "", line 1, in File "/Users/me/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 866, in runfile execfile(filename, namespace) File "/Users/me/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile builtins.execfile(filename, *where) File "/Users/me/Dropbox/Projects/dynamicLogrolling/code/04 - qualityFunction.py", line 100, in consensus_clustering_labels = CE.cluster_ensembles(MC) File "/Users/me/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/Cluster_Ensembles.py", line 301, in cluster_ensembles cluster_ensemble.append(consensus_functions[i](hdf5_file_name, cluster_runs, verbose, N_clusters_max)) File "/Users/me/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/Cluster_Ensembles.py", line 615, in CSPA return metis(hdf5_file_name, N_clusters_max) File "/Users/me/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/Cluster_Ensembles.py", line 938, in metis labels = sgraph(N_clusters_max, file_name) File "/Users/me/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/Cluster_Ensembles.py", line 1202, in sgraph with open(out_name, 'r') as file: IOError: [Errno 2] No such file or directory: 'wgraph_CSPA.part.370'

Any ideas?

[GGiecold-zz]

Check that gpmetis is up and running on your machine.

Gregory

On Oct 17, 2016 6:00 AM, "Omar Guerrero" notifications@github.com wrote:

Thanks a lot. This got me one step closer. Now I am getting the following error:

Traceback (most recent call last): File "", line 1, in File "/Users/me/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 866, in runfile execfile(filename, namespace) File "/Users/me/anaconda/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile builtins.execfile(filename, *where) File "/Users/me/Dropbox/Projects/dynamicLogrolling/code/04 - qualityFunction.py", line 100, in consensus_clustering_labels = CE.cluster_ensembles(MC) File "/Users/me/anaconda/lib/python2.7/site-packages/ Cluster_Ensembles/Cluster_Ensembles.py", line 301, in cluster_ensembles cluster_ensemble.append(consensus_functionsi http://hdf5_file_name,%20cluster_runs,%20verbose,%20N_clusters_max) File "/Users/me/anaconda/lib/python2.7/site-packages/ Cluster_Ensembles/Cluster_Ensembles.py", line 615, in CSPA return metis(hdf5_file_name, N_clusters_max) File "/Users/me/anaconda/lib/python2.7/site-packages/ Cluster_Ensembles/Cluster_Ensembles.py", line 938, in metis labels = sgraph(N_clusters_max, file_name) File "/Users/me/anaconda/lib/python2.7/site-packages/ Cluster_Ensembles/Cluster_Ensembles.py", line 1202, in sgraph with open(out_name, 'r') as file: IOError: [Errno 2] No such file or directory: 'wgraph_CSPA.part.370'

Any ideas?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GGiecold/Cluster_Ensembles/issues/5#issuecomment-254165145, or mute the thread https://github.com/notifications/unsubscribe-auth/AK3j57X9tgC66yHTvykBBxlRNcdoKwJ3ks5q00cngaJpZM4KYAWm .

[Jack-Lin-DS-AI]

Thanks Gregory. @oguerrer, here are what I did to make it work in Mac system:

1. Change 'memory' function in Cluster_Ensembles.py in Cluster_Ensembles packages (e.g., ~/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/Cluster_Ensembles.py) import psutil mem = psutil.virtual_memory() mem_info['free'] = mem.free mem_info['used'] = mem.used
2. Download METIS binary file 'gpmetis' from https://github.com/mjdousti/Squash/tree/master/metis/mac
3. Add the path (e.g., ~/bin) to gpmetis in .bash_profile export PATH="~/bin:$PATH"

[GGiecold-zz]

Exactly right.

Note that this issue doesn't arise for shmetis, which Cluster_Ensembles compiles on your machine.

Gregory

On Oct 21, 2016 5:03 PM, "Chih-Hsu Lin" notifications@github.com wrote:

Thanks Gregory. @oguerrer https://github.com/oguerrer, here are what I did to make it work in Mac system:

1. Change 'memory' function in Cluster_Ensembles.py in Cluster_Ensembles packages (e.g., ~/anaconda/lib/python2.7/site-packages/Cluster_Ensembles/ Cluster_Ensembles.py) import psutil mem = psutil.virtual_memory() mem_info['free'] = mem.free mem_info['used'] = mem.used
2. Download METIS binary file 'gpmetis' from https://github.com/mjdousti/ Squash/tree/master/metis/mac
3. Add the path (e.g., ~/bin) to gpmetis in .bash_profile export PATH="~/bin:$PATH"

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GGiecold/Cluster_Ensembles/issues/5#issuecomment-255464701, or mute the thread https://github.com/notifications/unsubscribe-auth/AK3j5ylKw34T1M-z0sP4_gRODenluE79ks5q2SizgaJpZM4KYAWm .

[isvanilin]

Thank you guys for the conversation - it helped me to run the library under OSX. Unfortunately I am having the same issue with the last error: FileNotFoundError: [Errno 2] No such file or directory: 'wgraph_CSPA.part.50'. I have experimented with different ways to add path to file gpmetis and finally Ive put it into usr/local/bin and now Im getting same error saying "Permission denied" both in terminal and in PyCharm.

Would appreciate any help!

[Jack-Lin-DS-AI]

You might need to change the permission of the file gpmetis. Go to the directory, e.g., cd usr/local/bin chmod 755 gpmetis

[isvanilin]

Thank you, ChihHsuLin! It worked.

[stshi8808]

Hello,.Thank you for helping.I am trying to run it in python 2.7 on windows,how can I make it work.I have no idea to get a solution. I'm a newer.

I'm looking forward to your help.Thanks a lot.

[GGiecold-zz]

As mentioned in Cluster_Ensemble's PyPI and GitHub documentations, only POSIX-compatible operating systems are supported.

Please try on Linux or OS X and let me know of any issue you might encounter then.

Gregory

On May 17, 2017 8:16 PM, "stshi8808" notifications@github.com wrote:

Hello,.Thank you for helping.I am trying to run it in python 2.7 on windows,how can I make it work.I have no idea to get a solution. I'm a newer.

I'm looking forward to your help.Thanks a lot.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/GGiecold/Cluster_Ensembles/issues/5#issuecomment-302265800, or mute the thread https://github.com/notifications/unsubscribe-auth/AK3j50pbQSFdGNcvbPtsTfW9FCeY0EB2ks5r643kgaJpZM4KYAWm .

[stshi8808]

Thanks a lot for helping me. When I use the package,I want to get each labels results of the samples with CSPA,HGPA,MCLA. I try to do like this as an example:

import numpy as np
import Cluster_Ensembles as CE
cluster_runs = np.random.randint(0, 8, (11, 50))
CSPA_clustering_labels = CE.CSPA(**hdf5_file_name**,cluster_runs, verbose = True, N_clusters_max = 8)
print CSPA_clustering_labels

But I don't know how to set the first Parameter(hdf5_file_name),because I don't have a hdf5_file.Could you give me the answer of above example?I am really newer in python. Thanks a lot.I'm looking forward to your help.Thanks a lot.

[GGiecold-zz]

Please refer to the documentation of PyTables for installation of this package, its dependencies (including the HDF5 C library and numexpr), as well as basic I/O operations with *.h5 files.

Gregory

On May 31, 2017 9:33 AM, "stshi8808" notifications@github.com wrote:

Thanks a lot for helping me. When I use the package,I want to get each labels results of the samples with CSPA,HGPA,MCLA. I try to do like this as an example:

import numpy as np import Cluster_Ensembles as CE cluster_runs = np.random.randint(0, 8, (11, 50)) CSPA_clustering_labels = CE.CSPA(hdf5_file_name,cluster_runs, verbose = True, N_clusters_max = 8) print CSPA_clustering_labels

But I don't know how to set the first Parameter(hdf5_file_name),because I don't have a hdf5_file.Could you give me the answer of above example?I am really newer in python. Thanks a lot.I'm looking forward to your help.Thanks a lot.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/GGiecold/Cluster_Ensembles/issues/5#issuecomment-305187774, or mute the thread https://github.com/notifications/unsubscribe-auth/AK3j5yD7qZXg2H9mpwl08cfxzx6m9tWfks5r_Ww3gaJpZM4KYAWm .

[stshi8808]

These days I tried to use Cluster_Ensembles package to do my paper. I want to get each labels results of the samples with CSPA,HGPA,MCLA. I try to do like this as an example:

import numpy as np
import Cluster_Ensembles as CE
import tables
cluster_runs = np.random.randint(0, 8, (15, 150))
hdf5_file_name = './Cluster_Ensembles.h5'
fileh = tables.open_file(hdf5_file_name, 'w')
fileh.create_group(fileh.root, 'consensus_group')
fileh.close()
hypergraph_adjacency = CE.build_hypergraph_adjacency(cluster_runs)
CE.store_hypergraph_adjacency(hypergraph_adjacency, hdf5_file_name)
consensus_clustering_labels = CE.HGPA(hdf5_file_name,cluster_runs, verbose = True, N_clusters_max = 8)

But there is an error:IOError: [Errno 2] No such file or directory: 'wgraph_HGPA.part.4' But the MCLA and CSPA are available to get the result,only HGPA can't work.Can you help me?

[GGiecold-zz]

Possibly with some more detail?

On Jun 1, 2017 2:02 AM, "stshi8808" notifications@github.com wrote:

These days I tried to use Cluster_Ensembles package to do my paper. I want to get each labels results of the samples with CSPA,HGPA,MCLA. I try to do like this as an example:

import numpy as np import Cluster_Ensembles as CE import tables cluster_runs = np.random.randint(0, 8, (15, 150)) hdf5_file_name = './Cluster_Ensembles.h5' fileh = tables.open_file(hdf5_file_name, 'w') fileh.create_group(fileh.root, 'consensus_group') fileh.close() hypergraph_adjacency = CE.build_hypergraph_adjacency(cluster_runs) CE.store_hypergraph_adjacency(hypergraph_adjacency, hdf5_file_name) consensus_clustering_labels = CE.HGPA(hdf5_file_name,cluster_runs, verbose = True, N_clusters_max = 8)

But there is an error:IOError: [Errno 2] No such file or directory: 'wgraph_HGPA.part.4' But the MCLA and CSPA are available to get the result,only HGPA can't work.Can you help me?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/GGiecold/Cluster_Ensembles/issues/5#issuecomment-305397087, or mute the thread https://github.com/notifications/unsubscribe-auth/AK3j56hXNU5wU6vtIImKHHEh6kw0md1_ks5r_lPogaJpZM4KYAWm .

[stshi8808]

Can you try the following code:

import numpy as np
import Cluster_Ensembles as CE
import tables
cluster_runs = np.random.randint(0, 8, (15, 150))
hdf5_file_name = './Cluster_Ensembles.h5'
fileh = tables.open_file(hdf5_file_name, 'w')
fileh.create_group(fileh.root, 'consensus_group')
fileh.close()
hypergraph_adjacency = CE.build_hypergraph_adjacency(cluster_runs)
CE.store_hypergraph_adjacency(hypergraph_adjacency, hdf5_file_name)
consensus_clustering_labels = CE.HGPA(hdf5_file_name,cluster_runs, verbose = True, N_clusters_max = 8)
print consensus_clustering_labels

The output : IOError: [Errno 2] No such file or directory: 'wgraph_HGPA.part.4' ,I don't know how to solve this problerm

[stshi8808]

Thanks Gregory,I try the these coede:

import numpy as np
import Cluster_Ensembles as CE
import tables
cluster_runs = np.random.randint(0, 8, (15, 150))
hdf5_file_name = './Cluster_Ensembles.h5'
fileh = tables.open_file(hdf5_file_name, 'w')
fileh.create_group(fileh.root, 'consensus_group')
fileh.close()
hypergraph_adjacency = CE.build_hypergraph_adjacency(cluster_runs)
CE.store_hypergraph_adjacency(hypergraph_adjacency, hdf5_file_name)
consensus_clustering_labels = CE.CSPA(hdf5_file_name,cluster_runs, verbose = True, N_clusters_max = 8)
print consensus_clustering_labels

It can work successfully. Also MCLA is also successful. Only HGPA has that error:No such file or directory: 'wgraph_HGPA.part.4' . Hope your reply, sorry to dsiturb.

[stshi8808]

The output is

*****
INFO: Cluster_Ensembles: HGPA: consensus clustering using HGPA.

#
INFO: Cluster_Ensembles: wgraph: writing wgraph_HGPA.
INFO: Cluster_Ensembles: wgraph: 15 vertices and 23 non-zero hyper-edges.
#

#
INFO: Cluster_Ensembles: sgraph: calling shmetis for hypergraph partitioning.
/bin/sh: 1: /home/shituo/Hypergraph_Partitioning/hmetis-1.5-linux/shmetis: not found
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/shituo/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 922, in runfile
    execfile(filename, namespace)
  File "/home/shituo/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 95, in execfile
    builtins.execfile(filename, *where)
  File "/home/shituo/crazy.py", line 1295, in <module>
    consensus_clustering_labels = HGPA(hdf5_file_name,cluster_runs, verbose = True, N_clusters_max = 4)
  File "/home/shituo/crazy.py", line 622, in HGPA
    return hmetis(hdf5_file_name, N_clusters_max)
  File "/home/shituo/crazy.py", line 947, in hmetis
    labels = sgraph(N_clusters_max, file_name)
  File "/home/shituo/crazy.py", line 1175, in sgraph
    with open(out_name, 'r') as file:
IOError: [Errno 2] No such file or directory: 'wgraph_HGPA.part.4'

[stshi8808]

Where can I get shmetis?

[stshi8808]

I find the shmetis, It can work now.Thanks a lot

[GGiecold-zz]

That was a common problem. Metis and shmetis are supposed to be compiled when installing Cluster_Ensemble. On some machines (including yours), they do not. Marco Sardelich posted a little fix some time ago on Cluster_Ensemble page on GitHub.

Glad it works now and all the best with your research.

Gregory

On Jun 1, 2017 6:56 AM, "stshi8808" notifications@github.com wrote:

I find the shmetis, It can work now.Thanks a lot

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/GGiecold/Cluster_Ensembles/issues/5#issuecomment-305460017, or mute the thread https://github.com/notifications/unsubscribe-auth/AK3j5wCMoH4j18I3kBc-36cpfwkKzDIvks5r_pjGgaJpZM4KYAWm .

[FatimaAnwer]

@GGiecold @ChihHsuLin @oguerrer @isvanilin @stshi8808 Please help me with the second error. I have tried so many ways to add path but I am still getting the error: IOError: [Errno 2] No such file or directory: 'wgraph_CSPA.part.50'

I am working on a Mac. I downloaded gpmetis.dms but I have no clue what to do with it. Should I run it or should I put it in the right directory. Please help.

[Lirunhua]

@FatimaAnwer you have to mv the 'gpmetis' file to dir: /usr/local/bin and 👍 chmod 755 gpmetis

[nafisegh]

@GGiecold @ChihHsuLin @oguerrer @isvanilin @stshi8808 Please help me. I want to use Cluster_Ensembles library for my research. I am using google colab, but when I use this command "consensus_clustering_labels=CE.cluster_ensembles(final_ensemble,N_clusters_max=int(np.max(final_ensemble)+1))"

I got this error:

---

INFO: Cluster_Ensembles: CSPA: consensus clustering using CSPA.

# INFO: Cluster_Ensembles: wgraph: writing wgraph_CSPA. #

# INFO: Cluster_Ensembles: sgraph: calling gpmetis for graph partitioning.

IOErrorTraceback (most recent call last)

in () 3 #from functools import reduce 4 ----> 5 consensus_clustering_labels=CE.cluster_ensembles(final_ensemble,N_clusters_max=int(np.max(final_ensemble)+1)) 3 frames /usr/local/lib/python2.7/dist-packages/Cluster_Ensembles/Cluster_Ensembles.pyc in sgraph(N_clusters_max, file_name) 1199 1200 labels = np.empty(0, dtype = int) -> 1201 with open(out_name, 'r') as file: 1202 print("INFO: Cluster_Ensembles: sgraph: (hyper)-graph partitioning completed; " 1203 "loading {}".format(out_name)) IOError: [Errno 2] No such file or directory: 'wgraph_CSPA.part.15' I read the last comments, but I am still confused about which file "gpmetis or shmetis" I must download and how can I add this file to a path in google colab?