search

baliga-lab / cmonkey2

Python port of cMonkey, a machine-learning based method for clustering
GNU Lesser General Public License v3.0
26 stars 16 forks source link

--use_BSCM is not working on my datset #53

Closed ABanaeiEsfahani closed 8 years ago

ABanaeiEsfahani commented 8 years ago

Hello,

I've tried cMonkey2 on my dataset with the following command but it gives me an error that you can see below. Could you please help me with this? Thanks.

./cmonkey.py --organism mtu --ratios ./example_data/Mtb/WayneAndTbX007_MedianNormalization_RatioByMeanD00.tsv --string ./example_data/Mtb/83332_v10.tsv --rsat_base_url http://pedagogix-tagc.univ-mrs.fr/rsat --rsat_organism Mycobacterium_tuberculosis_H37Rv_uid57777 --rsat_features gene --out ./Output_files/ --use_BSCM

2016-07-21 15:43:22 INFO checking MEME... 2016-07-21 15:43:22 INFO Input matrix has # rows: 2683, # columns: 33 2016-07-21 15:43:22 INFO # clusters/row: 2 2016-07-21 15:43:22 INFO # clusters/column: 179 2016-07-21 15:43:22 INFO # CLUSTERS: 268 2016-07-21 15:43:22 INFO use operons: 1 2016-07-21 15:43:22 INFO using MEME version 4.10.2 2016-07-21 15:43:23 INFO attempting automatic download of operons from Microbes Online 2016-07-21 15:43:23 INFO Loading STRING file at './example_data/Mtb/83332_v10.tsv' 2016-07-21 15:43:23 INFO KEGG = 'Mycobacterium tuberculosis H37Rv uid57777' -> RSAT = 'Mycobacterium_tuberculosis_H37Rv_uid57777' 2016-07-21 15:43:23 INFO Creating networks... 2016-07-21 15:43:23 INFO stringdb.read_edges2() 2016-07-21 15:43:24 INFO Finished loading ./example_data/Mtb/83332_v10.tsv 2016-07-21 15:43:25 INFO Processing network 5% 2016-07-21 15:43:25 INFO Processing network 10% 2016-07-21 15:43:25 INFO Processing network 15% 2016-07-21 15:43:25 INFO Processing network 20% 2016-07-21 15:43:25 INFO Processing network 25% 2016-07-21 15:43:25 INFO Processing network 30% 2016-07-21 15:43:25 INFO Processing network 35% 2016-07-21 15:43:25 INFO Processing network 40% 2016-07-21 15:43:25 INFO Processing network 45% 2016-07-21 15:43:25 INFO Processing network 50% 2016-07-21 15:43:25 INFO Processing network 55% 2016-07-21 15:43:25 INFO Processing network 60% 2016-07-21 15:43:25 INFO Processing network 65% 2016-07-21 15:43:25 INFO Processing network 70% 2016-07-21 15:43:25 INFO Processing network 75% 2016-07-21 15:43:25 INFO Processing network 80% 2016-07-21 15:43:25 INFO Processing network 85% 2016-07-21 15:43:26 INFO Processing network 90% 2016-07-21 15:43:26 INFO Processing network 95% 2016-07-21 15:43:26 INFO Processing network 100% 2016-07-21 15:43:26 WARNING 15 (out of 1247720) nodes not found in synonyms 2016-07-21 15:43:26 WARNING 1292 (out of 1247720) nodes not found in canonical gene names 2016-07-21 15:43:26 INFO stringdb.read_edges2(), 407198 edges read, 216662 edges ignored 2016-07-21 15:43:27 INFO MicrobesOnline - make_network() 2016-07-21 15:43:27 INFO MicrobesOnline.get_operon_predictions_for(83332) 2016-07-21 15:43:27 INFO 1302 prediction pairs read 2016-07-21 15:43:27 INFO 764 operons created 2016-07-21 15:43:27 INFO 2494 edges computed 2016-07-21 15:43:27 INFO Finished creating networks. 2016-07-21 15:43:28 INFO MicrobesOnline.get_operon_predictions_for(83332) 2016-07-21 15:43:28 INFO 1302 prediction pairs read 2016-07-21 15:43:29 INFO 764 operons created 2016-07-21 15:43:34 WARNING 1 genes not found in synonyms. 2016-07-21 15:43:34 INFO cmonkey_run config_params: 2016-07-21 15:43:34 INFO pipeline_file= 2016-07-21 15:43:34 INFO nomotifs=False 2016-07-21 15:43:34 INFO synonym_file=None 2016-07-21 15:43:34 INFO add_fuzz=rows 2016-07-21 15:43:34 INFO memb.max_cluster_rows_allowed=70 2016-07-21 15:43:34 INFO log_subresults=True 2016-07-21 15:43:34 INFO Rows={'scaling': ('scaling_const', 0.3), 'schedule': 1,2} 2016-07-21 15:43:34 INFO quantile_normalize=False 2016-07-21 15:43:34 INFO resume=False 2016-07-21 15:43:34 INFO debug=set() 2016-07-21 15:43:34 INFO new_data_file=False 2016-07-21 15:43:34 INFO rsat_features=gene 2016-07-21 15:43:34 INFO Motifs={'sequence_types': 'upstream', 'scaling': ('scaling_rvec', 'c(rep(1e-5, 100), seq(1e-5, 1, length=num_iterations_3/4))'), 'schedule': 2,10} 2016-07-21 15:43:34 INFO use_BSCM=True 2016-07-21 15:43:34 INFO ratios_file=./example_data/Mtb/WayneAndTbX007_MedianNormalization_RatioByMeanD00.tsv 2016-07-21 15:43:34 INFO output_dir=./Output_files/ 2016-07-21 15:43:34 INFO multiprocessing=True 2016-07-21 15:43:34 INFO use_chi2=False 2016-07-21 15:43:34 INFO scan_distances={'upstream': (-30, 250)} 2016-07-21 15:43:34 INFO Postprocessing={'run_tomtom': 'False'} 2016-07-21 15:43:34 INFO start_iteration=1 2016-07-21 15:43:34 INFO normalize_ratios=True 2016-07-21 15:43:34 INFO out_database=./Output_files/cmonkey_run.db 2016-07-21 15:43:34 INFO minimize_io=False 2016-07-21 15:43:34 INFO cache_dir=cache 2016-07-21 15:43:34 INFO case_sensitive=False 2016-07-21 15:43:34 INFO sequence_types=['upstream'] 2016-07-21 15:43:34 INFO num_iterations=2000 2016-07-21 15:43:34 INFO remap_network_nodes=False 2016-07-21 15:43:34 INFO memb.min_cluster_rows_allowed=3 2016-07-21 15:43:34 INFO memb.clusters_per_col=179 2016-07-21 15:43:34 INFO stats_freq=10 2016-07-21 15:43:34 INFO Networks={'scaling': ('scaling_rvec', 'seq(1e-5, 0.5, length=num_iterations_3/4)'), 'schedule': 1,7} 2016-07-21 15:43:34 INFO nonetworks=False 2016-07-21 15:43:34 INFO tmp_dir= 2016-07-21 15:43:34 INFO memb.clusters_per_row=2 2016-07-21 15:43:34 INFO operon_file=None 2016-07-21 15:43:34 INFO pipeline={'row-scoring': {'function': {'class': 'ScoringFunctionCombiner', 'module': 'cmonkey.scoring'}, 'id': 'combiner', 'args': {'functions': [{'function': {'class': 'RowScoringFunction', 'module': 'cmonkey.microarray'}, 'id': 'Rows'}, {'function': {'class': 'ScoringFunction', 'module': 'cmonkey.network'}, 'id': 'Networks'}, {'function': {'class': 'MemeScoringFunction', 'module': 'cmonkey.motif'}, 'id': 'Motifs'}]}}, 'column-scoring': {'function': {'class': 'ColumnScoringFunction', 'module': 'cmonkey.scoring'}, 'id': 'Columns'}} 2016-07-21 15:43:34 INFO checkratios=False 2016-07-21 15:43:34 INFO rsat_base_url=http://pedagogix-tagc.univ-mrs.fr/rsat 2016-07-21 15:43:34 INFO fasta_file=None 2016-07-21 15:43:34 INFO SequenceType-upstream={'search_distance': '-20,150', 'scan_distance': '-30,250'} 2016-07-21 15:43:34 INFO use_operons=True 2016-07-21 15:43:34 INFO num_clusters=268 2016-07-21 15:43:34 INFO ncbi_code=83332 2016-07-21 15:43:34 INFO interactive=False 2016-07-21 15:43:34 INFO dbfile_name=cmonkey_run.db 2016-07-21 15:43:34 INFO num_cores=None 2016-07-21 15:43:34 INFO MEME={'nmotifs_rvec': 'c(rep(1, num_iterations/3), rep(2, num_iterations/3))', 'background_order': '3', 'use_revcomp': 'True', 'schedule': 1,100, 'global_background': 'True', 'version': '4.10.2', 'max_width': '24', 'arg_mod': 'zoops'} 2016-07-21 15:43:34 INFO Weeder={'orgcode': '', 'nmotifs_rvec': 'c(rep(1, num_iterations/3), rep(2, num_iterations/3))', 'analysis': 'small', 'freqfile_dir': '', 'global_background': 'True', 'schedule': 1,100, 'top': '50'} 2016-07-21 15:43:34 INFO search_distances={'upstream': (-20, 150)} 2016-07-21 15:43:34 INFO logfile=None 2016-07-21 15:43:34 INFO organism_code=mtu 2016-07-21 15:43:34 INFO postadjust=True 2016-07-21 15:43:34 INFO command_line=./cmonkey.py --organism mtu --ratios ./example_data/Mtb/WayneAndTbX007_MedianNormalization_RatioByMeanD00.tsv --string ./example_data/Mtb/83332_v10.tsv --rsat_base_url http://pedagogix-tagc.univ-mrs.fr/rsat --rsat_organism Mycobacterium_tuberculosis_H37Rv_uid57777 --rsat_features gene --out ./Output_files/ --use_BSCM 2016-07-21 15:43:34 INFO rsat_organism=Mycobacterium_tuberculosis_H37Rv_uid57777 2016-07-21 15:43:34 INFO random_seed=229787200 2016-07-21 15:43:34 INFO string_file=./example_data/Mtb/83332_v10.tsv 2016-07-21 15:43:34 INFO memb.max_changes_per_col=5 2016-07-21 15:43:34 INFO debug_freq=50 2016-07-21 15:43:34 INFO use_string=True 2016-07-21 15:43:34 INFO memb.prob_row_change=0.5 2016-07-21 15:43:34 INFO rsat_dir=None 2016-07-21 15:43:34 INFO memb.max_changes_per_row=1 2016-07-21 15:43:34 INFO Columns={'schedule': 1,5} 2016-07-21 15:43:34 INFO memb.prob_col_change=1.0 2016-07-21 15:43:34 INFO result_freq=10 2016-07-21 15:43:34 INFO Iteration # 1 2016-07-21 15:43:48 INFO running meme/mast on cluster 1, # sequences: 21 2016-07-21 15:43:48 INFO running meme/mast on cluster 18, # sequences: 11 2016-07-21 15:43:48 INFO running meme/mast on cluster 35, # sequences: 23 2016-07-21 15:43:48 INFO running meme/mast on cluster 52, # sequences: 5 2016-07-21 15:43:49 INFO running meme/mast on cluster 53, # sequences: 7 2016-07-21 15:43:50 INFO running meme/mast on cluster 54, # sequences: 3 2016-07-21 15:43:50 INFO running meme/mast on cluster 19, # sequences: 8 2016-07-21 15:43:50 INFO running meme/mast on cluster 55, # sequences: 5 2016-07-21 15:43:51 INFO running meme/mast on cluster 20, # sequences: 15 2016-07-21 15:43:51 INFO running meme/mast on cluster 56, # sequences: 7 2016-07-21 15:43:52 INFO running meme/mast on cluster 57, # sequences: 12 2016-07-21 15:43:52 INFO running meme/mast on cluster 2, # sequences: 7 2016-07-21 15:43:52 INFO running meme/mast on cluster 36, # sequences: 6 2016-07-21 15:43:53 INFO running meme/mast on cluster 3, # sequences: 9 2016-07-21 15:43:53 INFO running meme/mast on cluster 37, # sequences: 7 2016-07-21 15:43:53 INFO running meme/mast on cluster 21, # sequences: 23 2016-07-21 15:43:53 INFO running meme/mast on cluster 58, # sequences: 11 2016-07-21 15:43:53 INFO running meme/mast on cluster 38, # sequences: 3 2016-07-21 15:43:54 INFO running meme/mast on cluster 4, # sequences: 13 2016-07-21 15:43:54 INFO running meme/mast on cluster 39, # sequences: 6 2016-07-21 15:43:55 INFO running meme/mast on cluster 40, # sequences: 16 2016-07-21 15:43:55 INFO running meme/mast on cluster 59, # sequences: 14 2016-07-21 15:43:55 INFO running meme/mast on cluster 5, # sequences: 15 2016-07-21 15:43:57 INFO running meme/mast on cluster 22, # sequences: 9 2016-07-21 15:43:57 INFO running meme/mast on cluster 60, # sequences: 13 2016-07-21 15:43:57 INFO running meme/mast on cluster 41, # sequences: 2 2016-07-21 15:43:57 INFO running meme/mast on cluster 42, # sequences: 8 2016-07-21 15:43:58 INFO running meme/mast on cluster 6, # sequences: 9 2016-07-21 15:43:58 INFO running meme/mast on cluster 23, # sequences: 3 2016-07-21 15:43:58 INFO running meme/mast on cluster 43, # sequences: 6 2016-07-21 15:43:58 INFO running meme/mast on cluster 24, # sequences: 7 2016-07-21 15:43:59 INFO running meme/mast on cluster 61, # sequences: 16 2016-07-21 15:43:59 INFO running meme/mast on cluster 7, # sequences: 7 2016-07-21 15:43:59 INFO running meme/mast on cluster 44, # sequences: 11 2016-07-21 15:43:59 INFO running meme/mast on cluster 25, # sequences: 3 2016-07-21 15:44:00 INFO running meme/mast on cluster 8, # sequences: 14 2016-07-21 15:44:00 INFO running meme/mast on cluster 26, # sequences: 7 2016-07-21 15:44:01 INFO running meme/mast on cluster 45, # sequences: 5 2016-07-21 15:44:01 INFO running meme/mast on cluster 27, # sequences: 7 2016-07-21 15:44:01 INFO running meme/mast on cluster 62, # sequences: 4 2016-07-21 15:44:01 INFO running meme/mast on cluster 46, # sequences: 6 2016-07-21 15:44:01 INFO running meme/mast on cluster 9, # sequences: 7 2016-07-21 15:44:01 INFO running meme/mast on cluster 63, # sequences: 6 2016-07-21 15:44:02 INFO running meme/mast on cluster 28, # sequences: 6 2016-07-21 15:44:02 INFO running meme/mast on cluster 47, # sequences: 19 2016-07-21 15:44:02 INFO running meme/mast on cluster 10, # sequences: 5 2016-07-21 15:44:02 INFO running meme/mast on cluster 64, # sequences: 24 2016-07-21 15:44:02 INFO running meme/mast on cluster 29, # sequences: 10 2016-07-21 15:44:03 INFO running meme/mast on cluster 11, # sequences: 15 2016-07-21 15:44:04 INFO running meme/mast on cluster 30, # sequences: 7 2016-07-21 15:44:05 INFO running meme/mast on cluster 31, # sequences: 50 2016-07-21 15:44:05 INFO running meme/mast on cluster 12, # sequences: 18 2016-07-21 15:44:05 INFO running meme/mast on cluster 48, # sequences: 19 2016-07-21 15:44:06 INFO running meme/mast on cluster 65, # sequences: 13 2016-07-21 15:44:08 INFO running meme/mast on cluster 13, # sequences: 8 2016-07-21 15:44:08 INFO running meme/mast on cluster 66, # sequences: 4 2016-07-21 15:44:08 INFO running meme/mast on cluster 49, # sequences: 3 2016-07-21 15:44:09 INFO running meme/mast on cluster 14, # sequences: 6 2016-07-21 15:44:09 INFO running meme/mast on cluster 50, # sequences: 16 2016-07-21 15:44:09 INFO running meme/mast on cluster 67, # sequences: 9 2016-07-21 15:44:09 INFO running meme/mast on cluster 15, # sequences: 17 2016-07-21 15:44:10 INFO running meme/mast on cluster 68, # sequences: 6 2016-07-21 15:44:11 INFO running meme/mast on cluster 51, # sequences: 3 2016-07-21 15:44:11 INFO running meme/mast on cluster 69, # sequences: 4 2016-07-21 15:44:11 INFO running meme/mast on cluster 86, # sequences: 4 2016-07-21 15:44:12 INFO running meme/mast on cluster 70, # sequences: 2 2016-07-21 15:44:12 INFO running meme/mast on cluster 71, # sequences: 10 2016-07-21 15:44:12 INFO running meme/mast on cluster 87, # sequences: 14 2016-07-21 15:44:12 INFO running meme/mast on cluster 16, # sequences: 4 2016-07-21 15:44:13 INFO running meme/mast on cluster 17, # sequences: 12 2016-07-21 15:44:13 INFO running meme/mast on cluster 72, # sequences: 16 2016-07-21 15:44:14 INFO running meme/mast on cluster 88, # sequences: 20 2016-07-21 15:44:14 INFO running meme/mast on cluster 103, # sequences: 6 2016-07-21 15:44:15 INFO running meme/mast on cluster 104, # sequences: 13 2016-07-21 15:44:15 INFO running meme/mast on cluster 73, # sequences: 8 2016-07-21 15:44:16 INFO running meme/mast on cluster 74, # sequences: 15 2016-07-21 15:44:17 INFO running meme/mast on cluster 105, # sequences: 14 2016-07-21 15:44:17 INFO running meme/mast on cluster 32, # sequences: 10 2016-07-21 15:44:17 INFO running meme/mast on cluster 89, # sequences: 15 2016-07-21 15:44:18 INFO running meme/mast on cluster 75, # sequences: 6 2016-07-21 15:44:19 INFO running meme/mast on cluster 33, # sequences: 5 2016-07-21 15:44:19 INFO running meme/mast on cluster 106, # sequences: 9 2016-07-21 15:44:19 INFO running meme/mast on cluster 76, # sequences: 5 2016-07-21 15:44:19 INFO running meme/mast on cluster 34, # sequences: 7 2016-07-21 15:44:19 INFO running meme/mast on cluster 90, # sequences: 12 2016-07-21 15:44:20 INFO running meme/mast on cluster 77, # sequences: 7 2016-07-21 15:44:20 INFO running meme/mast on cluster 107, # sequences: 8 2016-07-21 15:44:20 INFO running meme/mast on cluster 120, # sequences: 4 2016-07-21 15:44:21 INFO running meme/mast on cluster 78, # sequences: 14 2016-07-21 15:44:21 INFO running meme/mast on cluster 121, # sequences: 20 2016-07-21 15:44:21 INFO running meme/mast on cluster 91, # sequences: 15 2016-07-21 15:44:21 INFO running meme/mast on cluster 108, # sequences: 7 2016-07-21 15:44:22 INFO running meme/mast on cluster 109, # sequences: 6 2016-07-21 15:44:22 INFO running meme/mast on cluster 79, # sequences: 10 2016-07-21 15:44:23 INFO running meme/mast on cluster 92, # sequences: 8 2016-07-21 15:44:23 INFO running meme/mast on cluster 110, # sequences: 2 2016-07-21 15:44:23 INFO running meme/mast on cluster 111, # sequences: 7 2016-07-21 15:44:24 INFO running meme/mast on cluster 112, # sequences: 5 2016-07-21 15:44:24 INFO running meme/mast on cluster 80, # sequences: 2 2016-07-21 15:44:24 INFO running meme/mast on cluster 81, # sequences: 7 2016-07-21 15:44:24 INFO running meme/mast on cluster 93, # sequences: 18 2016-07-21 15:44:24 INFO running meme/mast on cluster 122, # sequences: 16 2016-07-21 15:44:24 INFO running meme/mast on cluster 113, # sequences: 4 2016-07-21 15:44:25 INFO running meme/mast on cluster 82, # sequences: 3 2016-07-21 15:44:25 INFO running meme/mast on cluster 114, # sequences: 8 2016-07-21 15:44:25 INFO running meme/mast on cluster 83, # sequences: 8 2016-07-21 15:44:26 INFO running meme/mast on cluster 115, # sequences: 11 2016-07-21 15:44:26 INFO running meme/mast on cluster 84, # sequences: 39 2016-07-21 15:44:26 INFO running meme/mast on cluster 123, # sequences: 4 2016-07-21 15:44:27 INFO running meme/mast on cluster 94, # sequences: 2 2016-07-21 15:44:27 INFO running meme/mast on cluster 95, # sequences: 3 2016-07-21 15:44:27 INFO running meme/mast on cluster 124, # sequences: 10 2016-07-21 15:44:27 INFO running meme/mast on cluster 96, # sequences: 5 2016-07-21 15:44:27 INFO running meme/mast on cluster 116, # sequences: 12 2016-07-21 15:44:28 INFO running meme/mast on cluster 97, # sequences: 7 2016-07-21 15:44:28 INFO running meme/mast on cluster 125, # sequences: 10 2016-07-21 15:44:28 INFO running meme/mast on cluster 98, # sequences: 14 2016-07-21 15:44:29 INFO running meme/mast on cluster 117, # sequences: 6 2016-07-21 15:44:29 INFO running meme/mast on cluster 126, # sequences: 7 2016-07-21 15:44:30 INFO running meme/mast on cluster 118, # sequences: 2 2016-07-21 15:44:30 INFO running meme/mast on cluster 119, # sequences: 7 2016-07-21 15:44:30 INFO running meme/mast on cluster 127, # sequences: 18 2016-07-21 15:44:30 INFO running meme/mast on cluster 99, # sequences: 9 2016-07-21 15:44:31 INFO running meme/mast on cluster 137, # sequences: 26 2016-07-21 15:44:32 INFO running meme/mast on cluster 100, # sequences: 5 2016-07-21 15:44:32 INFO running meme/mast on cluster 101, # sequences: 15 2016-07-21 15:44:34 INFO running meme/mast on cluster 128, # sequences: 7 2016-07-21 15:44:34 INFO running meme/mast on cluster 129, # sequences: 9 2016-07-21 15:44:35 INFO running meme/mast on cluster 102, # sequences: 10 2016-07-21 15:44:35 INFO running meme/mast on cluster 85, # sequences: 19 2016-07-21 15:44:36 INFO running meme/mast on cluster 138, # sequences: 6 2016-07-21 15:44:36 INFO running meme/mast on cluster 130, # sequences: 4 2016-07-21 15:44:36 INFO running meme/mast on cluster 154, # sequences: 6 2016-07-21 15:44:36 INFO running meme/mast on cluster 139, # sequences: 13 2016-07-21 15:44:37 INFO running meme/mast on cluster 131, # sequences: 12 2016-07-21 15:44:37 INFO running meme/mast on cluster 155, # sequences: 4 2016-07-21 15:44:37 INFO running meme/mast on cluster 156, # sequences: 20 2016-07-21 15:44:38 INFO running meme/mast on cluster 140, # sequences: 14 2016-07-21 15:44:38 INFO running meme/mast on cluster 132, # sequences: 10 2016-07-21 15:44:39 INFO running meme/mast on cluster 171, # sequences: 11 2016-07-21 15:44:40 INFO running meme/mast on cluster 133, # sequences: 10 2016-07-21 15:44:40 INFO running meme/mast on cluster 141, # sequences: 6 2016-07-21 15:44:40 INFO running meme/mast on cluster 172, # sequences: 7 2016-07-21 15:44:41 INFO running meme/mast on cluster 157, # sequences: 5 2016-07-21 15:44:41 INFO running meme/mast on cluster 142, # sequences: 10 2016-07-21 15:44:41 INFO running meme/mast on cluster 134, # sequences: 5 2016-07-21 15:44:41 INFO running meme/mast on cluster 173, # sequences: 7 2016-07-21 15:44:42 INFO running meme/mast on cluster 158, # sequences: 19 2016-07-21 15:44:42 INFO running meme/mast on cluster 135, # sequences: 11 2016-07-21 15:44:42 INFO running meme/mast on cluster 174, # sequences: 11 2016-07-21 15:44:42 INFO running meme/mast on cluster 143, # sequences: 3 2016-07-21 15:44:43 INFO running meme/mast on cluster 144, # sequences: 5 2016-07-21 15:44:44 INFO running meme/mast on cluster 175, # sequences: 21 2016-07-21 15:44:44 INFO running meme/mast on cluster 145, # sequences: 2 2016-07-21 15:44:44 INFO running meme/mast on cluster 146, # sequences: 6 2016-07-21 15:44:44 INFO running meme/mast on cluster 136, # sequences: 4 2016-07-21 15:44:45 INFO running meme/mast on cluster 188, # sequences: 7 2016-07-21 15:44:45 INFO running meme/mast on cluster 147, # sequences: 10 2016-07-21 15:44:45 INFO running meme/mast on cluster 159, # sequences: 14 2016-07-21 15:44:45 INFO running meme/mast on cluster 189, # sequences: 7 2016-07-21 15:44:46 INFO running meme/mast on cluster 148, # sequences: 5 2016-07-21 15:44:46 INFO running meme/mast on cluster 190, # sequences: 5 2016-07-21 15:44:47 INFO running meme/mast on cluster 160, # sequences: 8 2016-07-21 15:44:47 INFO running meme/mast on cluster 149, # sequences: 5 2016-07-21 15:44:47 INFO running meme/mast on cluster 191, # sequences: 9 2016-07-21 15:44:47 INFO running meme/mast on cluster 176, # sequences: 6 2016-07-21 15:44:48 INFO running meme/mast on cluster 150, # sequences: 8 2016-07-21 15:44:48 INFO running meme/mast on cluster 161, # sequences: 10 2016-07-21 15:44:48 INFO running meme/mast on cluster 177, # sequences: 5 2016-07-21 15:44:48 INFO running meme/mast on cluster 192, # sequences: 12 2016-07-21 15:44:49 INFO running meme/mast on cluster 151, # sequences: 4 2016-07-21 15:44:49 INFO running meme/mast on cluster 178, # sequences: 8 2016-07-21 15:44:49 INFO running meme/mast on cluster 152, # sequences: 80 2016-07-21 15:44:49 INFO running meme/mast on cluster 153, # sequences: 9 2016-07-21 15:44:49 INFO running meme/mast on cluster 162, # sequences: 12 2016-07-21 15:44:50 INFO running meme/mast on cluster 179, # sequences: 5 2016-07-21 15:44:50 INFO running meme/mast on cluster 193, # sequences: 3 2016-07-21 15:44:50 INFO running meme/mast on cluster 205, # sequences: 11 2016-07-21 15:44:51 INFO running meme/mast on cluster 194, # sequences: 18 2016-07-21 15:44:51 INFO running meme/mast on cluster 180, # sequences: 34 2016-07-21 15:44:51 INFO running meme/mast on cluster 163, # sequences: 7 2016-07-21 15:44:52 INFO running meme/mast on cluster 206, # sequences: 6 2016-07-21 15:44:52 INFO running meme/mast on cluster 164, # sequences: 24 2016-07-21 15:44:53 INFO running meme/mast on cluster 207, # sequences: 5 2016-07-21 15:44:54 INFO running meme/mast on cluster 208, # sequences: 13 2016-07-21 15:44:54 INFO running meme/mast on cluster 195, # sequences: 3 2016-07-21 15:44:54 INFO running meme/mast on cluster 196, # sequences: 5 2016-07-21 15:44:55 INFO running meme/mast on cluster 197, # sequences: 8 2016-07-21 15:44:55 INFO running meme/mast on cluster 209, # sequences: 8 2016-07-21 15:44:56 INFO running meme/mast on cluster 198, # sequences: 6 2016-07-21 15:44:56 INFO running meme/mast on cluster 165, # sequences: 8 2016-07-21 15:44:57 INFO running meme/mast on cluster 210, # sequences: 6 2016-07-21 15:44:57 INFO running meme/mast on cluster 199, # sequences: 6 2016-07-21 15:44:57 INFO running meme/mast on cluster 211, # sequences: 13 2016-07-21 15:44:57 INFO running meme/mast on cluster 166, # sequences: 15 2016-07-21 15:44:58 INFO running meme/mast on cluster 200, # sequences: 4 2016-07-21 15:44:59 INFO running meme/mast on cluster 181, # sequences: 14 2016-07-21 15:44:59 INFO running meme/mast on cluster 201, # sequences: 6 2016-07-21 15:44:59 INFO running meme/mast on cluster 212, # sequences: 3 2016-07-21 15:44:59 INFO running meme/mast on cluster 213, # sequences: 8 2016-07-21 15:44:59 INFO running meme/mast on cluster 202, # sequences: 4 2016-07-21 15:45:00 INFO running meme/mast on cluster 167, # sequences: 3 2016-07-21 15:45:00 INFO running meme/mast on cluster 168, # sequences: 4 2016-07-21 15:45:00 INFO running meme/mast on cluster 203, # sequences: 5 2016-07-21 15:45:00 INFO running meme/mast on cluster 214, # sequences: 15 2016-07-21 15:45:00 INFO running meme/mast on cluster 169, # sequences: 19 2016-07-21 15:45:01 INFO running meme/mast on cluster 182, # sequences: 4 2016-07-21 15:45:01 INFO running meme/mast on cluster 204, # sequences: 5 2016-07-21 15:45:01 INFO running meme/mast on cluster 183, # sequences: 7 2016-07-21 15:45:02 INFO running meme/mast on cluster 222, # sequences: 9 2016-07-21 15:45:02 INFO running meme/mast on cluster 184, # sequences: 6 2016-07-21 15:45:03 INFO running meme/mast on cluster 215, # sequences: 5 2016-07-21 15:45:03 INFO running meme/mast on cluster 185, # sequences: 2 2016-07-21 15:45:03 INFO running meme/mast on cluster 186, # sequences: 6 2016-07-21 15:45:03 INFO running meme/mast on cluster 223, # sequences: 10 2016-07-21 15:45:03 INFO running meme/mast on cluster 216, # sequences: 21 2016-07-21 15:45:04 INFO running meme/mast on cluster 170, # sequences: 6 2016-07-21 15:45:04 INFO running meme/mast on cluster 187, # sequences: 3 2016-07-21 15:45:04 INFO running meme/mast on cluster 224, # sequences: 6 2016-07-21 15:45:04 INFO running meme/mast on cluster 239, # sequences: 18 2016-07-21 15:45:04 INFO running meme/mast on cluster 256, # sequences: 13 2016-07-21 15:45:05 INFO running meme/mast on cluster 225, # sequences: 21 2016-07-21 15:45:06 INFO running meme/mast on cluster 257, # sequences: 12 2016-07-21 15:45:07 INFO running meme/mast on cluster 217, # sequences: 12 2016-07-21 15:45:08 INFO running meme/mast on cluster 240, # sequences: 6 2016-07-21 15:45:08 INFO running meme/mast on cluster 258, # sequences: 8 2016-07-21 15:45:08 INFO running meme/mast on cluster 241, # sequences: 6 2016-07-21 15:45:08 INFO running meme/mast on cluster 226, # sequences: 7 2016-07-21 15:45:09 INFO running meme/mast on cluster 259, # sequences: 14 2016-07-21 15:45:09 INFO running meme/mast on cluster 242, # sequences: 6 2016-07-21 15:45:09 INFO running meme/mast on cluster 218, # sequences: 26 2016-07-21 15:45:09 INFO running meme/mast on cluster 227, # sequences: 5 2016-07-21 15:45:10 INFO running meme/mast on cluster 243, # sequences: 3 2016-07-21 15:45:10 INFO running meme/mast on cluster 228, # sequences: 11 2016-07-21 15:45:10 INFO running meme/mast on cluster 244, # sequences: 3 2016-07-21 15:45:11 INFO running meme/mast on cluster 245, # sequences: 9 2016-07-21 15:45:11 INFO running meme/mast on cluster 260, # sequences: 23 2016-07-21 15:45:11 INFO running meme/mast on cluster 229, # sequences: 5 2016-07-21 15:45:12 INFO running meme/mast on cluster 246, # sequences: 7 2016-07-21 15:45:12 INFO running meme/mast on cluster 230, # sequences: 4 2016-07-21 15:45:13 INFO running meme/mast on cluster 231, # sequences: 10 2016-07-21 15:45:13 INFO running meme/mast on cluster 247, # sequences: 8 2016-07-21 15:45:14 INFO running meme/mast on cluster 219, # sequences: 23 2016-07-21 15:45:14 INFO running meme/mast on cluster 248, # sequences: 6 2016-07-21 15:45:14 INFO running meme/mast on cluster 232, # sequences: 8 2016-07-21 15:45:15 INFO running meme/mast on cluster 249, # sequences: 4 2016-07-21 15:45:15 INFO running meme/mast on cluster 261, # sequences: 6 2016-07-21 15:45:15 INFO running meme/mast on cluster 233, # sequences: 4 2016-07-21 15:45:15 INFO running meme/mast on cluster 250, # sequences: 8 2016-07-21 15:45:16 INFO running meme/mast on cluster 262, # sequences: 37 2016-07-21 15:45:16 INFO running meme/mast on cluster 234, # sequences: 7 2016-07-21 15:45:16 INFO running meme/mast on cluster 251, # sequences: 3 2016-07-21 15:45:17 INFO running meme/mast on cluster 235, # sequences: 15 2016-07-21 15:45:17 INFO running meme/mast on cluster 252, # sequences: 9 2016-07-21 15:45:18 INFO running meme/mast on cluster 220, # sequences: 6 2016-07-21 15:45:18 INFO running meme/mast on cluster 253, # sequences: 7 2016-07-21 15:45:19 INFO running meme/mast on cluster 236, # sequences: 4 2016-07-21 15:45:19 INFO running meme/mast on cluster 221, # sequences: 7 2016-07-21 15:45:19 INFO running meme/mast on cluster 237, # sequences: 8 2016-07-21 15:45:19 INFO running meme/mast on cluster 254, # sequences: 9 2016-07-21 15:45:20 INFO running meme/mast on cluster 238, # sequences: 7 2016-07-21 15:45:20 INFO running meme/mast on cluster 255, # sequences: 12 2016-07-21 15:45:23 INFO running meme/mast on cluster 263, # sequences: 4 2016-07-21 15:45:23 INFO running meme/mast on cluster 264, # sequences: 11 2016-07-21 15:45:24 INFO running meme/mast on cluster 265, # sequences: 7 2016-07-21 15:45:25 INFO running meme/mast on cluster 266, # sequences: 12 2016-07-21 15:45:26 INFO running meme/mast on cluster 267, # sequences: 5 2016-07-21 15:45:26 INFO running meme/mast on cluster 268, # sequences: 4 2016-07-21 15:45:27 INFO Calculating some backgrounds for about 21 genes Traceback (most recent call last): File "./cmonkey.py", line 36, in cmonkey_run.run() File "/home/amir/Others/cmonkey2/cmonkey/cmonkey_run.py", line 516, in run self.run_iterations() File "/home/amir/Others/cmonkey2/cmonkey/cmonkey_run.py", line 774, in run_iterations self.run_iteration(iteration, force=force) File "/home/amir/Others/cmonkey2/cmonkey/cmonkey_run.py", line 710, in run_iteration cscores = self.column_scoring.compute(iteration_result) File "/home/amir/Others/cmonkey2/cmonkey/scoring.py", line 134, in compute reference_matrix) File "/home/amir/Others/cmonkey2/cmonkey/scoring.py", line 233, in do_compute self.BSCM_obj) File "/home/amir/Others/cmonkey2/cmonkey/scoring.py", line 282, in compute_column_scores cur_column_scores = BSCM_obj.getPvals(make_submatrix(cluster).row_names, num_cores=num_cores) File "/home/amir/Others/cmonkey2/cmonkey/BSCM.py", line 186, in getPvals self.allVars[cn][curN] = newVars[idx] TypeError: 'map' object is not subscriptable

weiju commented 8 years ago

thanks for reporting the problem, this is was a missing list call under Python 3. I have pushed the fix now

ABanaeiEsfahani commented 8 years ago

Thanks but I can't work with the new version anymore. I get the following error. I guess it can't recognize my python3. right?

:~/Others/cmonkey2$ ./bin/cmonkey2 --help Traceback (most recent call last): File "/home/amir/Others/cmonkey2/bin/cmonkey2.py", line 18, in args, params, ratios = conf.setup() File "/home/amir/Others/cmonkey2/cmonkey/config.py", line 288, in setup config_parser = get_config_parser() File "/home/amir/Others/cmonkey2/cmonkey/config.py", line 193, in __get_config_parser config_path = resource_filename(Requirement.parse("cmonkey2"), USER_INI_PATH) File "/usr/local/lib/python3.5/site-packages/pkg_resources/__init.py", line 1171, in resource_filename return get_provider(package_or_requirement).get_resource_filename( File "/usr/local/lib/python3.5/site-packages/pkg_resources/init.py", line 415, in get_provider return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] File "/usr/local/lib/python3.5/site-packages/pkg_resources/init.py", line 943, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/local/lib/python3.5/site-packages/pkg_resources/init.py", line 829, in resolve raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'cmonkey2' distribution was not found and is required by the application

weiju commented 8 years ago

Yes, I am currently restructuring cmonkey2 so it will be able to install through pip or setup tools. It seems the configuration loading was broken while I did that change and it slipped through during my testing. This issue should be fixed now though. Thanks for letting me know.

ABanaeiEsfahani commented 8 years ago

Thanks for taking care of this. But unfortunately I faced a new error. Could you please look at this as well?

./bin/cmonkey2 ./example_data/Mtb/WayneAndTbX007_MedianNormalization_RatioByMeanD00.tsv --organism mtu --string ./example_data/Mtb/83332_v10.tsv --rsat_base_url http://pedagogix-tagc.univ-mrs.fr/rsat --rsat_organism Mycobacterium_tuberculosis_H37Rv_uid57777 --rsat_features gene --out ./Output_files/ --use_BSCM 2016-08-08 10:10:45 INFO checking MEME... 2016-08-08 10:10:46 INFO Input matrix has # rows: 2683, # columns: 33 2016-08-08 10:10:46 INFO # clusters/row: 2 2016-08-08 10:10:46 INFO # clusters/column: 179 2016-08-08 10:10:46 INFO # CLUSTERS: 268 2016-08-08 10:10:46 INFO use operons: 1 2016-08-08 10:10:46 INFO using MEME version 4.10.2 2016-08-08 10:10:48 INFO attempting automatic download of operons from Microbes Online 2016-08-08 10:10:48 INFO Loading STRING file at './example_data/Mtb/83332_v10.tsv' 2016-08-08 10:10:48 INFO KEGG = 'Mycobacterium tuberculosis H37Rv uid57777' -> RSAT = 'Mycobacterium_tuberculosis_H37Rv_uid57777' 2016-08-08 10:10:48 INFO Creating networks... 2016-08-08 10:10:48 INFO stringdb.read_edges2() 2016-08-08 10:10:53 INFO Finished loading ./example_data/Mtb/83332_v10.tsv 2016-08-08 10:10:53 INFO Processing network 5% 2016-08-08 10:10:53 INFO Processing network 10% 2016-08-08 10:10:53 INFO Processing network 15% 2016-08-08 10:10:53 INFO Processing network 20% 2016-08-08 10:10:54 INFO Processing network 25% 2016-08-08 10:10:54 INFO Processing network 30% 2016-08-08 10:10:54 INFO Processing network 35% 2016-08-08 10:10:54 INFO Processing network 40% 2016-08-08 10:10:54 INFO Processing network 45% 2016-08-08 10:10:54 INFO Processing network 50% 2016-08-08 10:10:54 INFO Processing network 55% 2016-08-08 10:10:54 INFO Processing network 60% 2016-08-08 10:10:55 INFO Processing network 65% 2016-08-08 10:10:55 INFO Processing network 70% 2016-08-08 10:10:55 INFO Processing network 75% 2016-08-08 10:10:55 INFO Processing network 80% 2016-08-08 10:10:55 INFO Processing network 85% 2016-08-08 10:10:55 INFO Processing network 90% 2016-08-08 10:10:55 INFO Processing network 95% 2016-08-08 10:10:55 INFO Processing network 100% 2016-08-08 10:10:55 WARNING 3967 (out of 1247720) nodes not found in synonyms 2016-08-08 10:10:55 INFO stringdb.read_edges2(), 0 edges read, 623860 edges ignored Traceback (most recent call last): File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/bin/cmonkey2.py", line 36, in cmonkey_run.run() File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/cmonkey/cmonkey_run.py", line 512, in run self.prepare_run() File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/cmonkey/cmonkey_run.py", line 474, in prepare_run thesaurus = self.organism().thesaurus() File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/cmonkey/cmonkey_run.py", line 231, in organism self.organism = self.make_organism() File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/cmonkey/cmonkey_run.py", line 341, in make_organism self['fasta_file']) File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/cmonkey/organism.py", line 244, in __init fasta_file) File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/cmonkey/organism.py", line 117, in init OrganismBase.init(self, code, network_factories, ratios=ratios) File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/cmonkey/organism.py", line 72, in init self.__networks.append(make_network(self, ratios)) File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/cmonkey/stringdb.py", line 135, in make_network organism, ratios) File "/IMSB/ra/bamir/Computation_on_Clusters/My_virtual_env/Others/cmonkey2/cmonkey/network.py", line 150, in create raise Exception("Error: only %d edges in network '%s'" % (len(network_edges), name)) Exception: Error: only 0 edges in network 'STRING'

weiju commented 8 years ago

This error usually indicates that there are problems somewhere in the naming, in our lab, we had this error with a tuberculosis input set because the gene names in our input ratios were in uppercase, while the STRING and RSAT gene names had mixed case, could you please check, whether

Thanks