erkiaun
commented
3 years ago Hello!
Having fewer threads than 8 shouldn't be a problem for glistmaker. Therefore, it is strange that command "glistmaker VL_0004.fasta -w 13 -o genome1 --num_threads 4" works, but "glistmaker VL_0004.fasta -w 13 -o genome1" does not.
The error, however, seems to be due to the temporary mistake, where different versions of glistmaker and glistcompare binaries were in the bin.
Hello, I have a problem running PhenotypeSeeker.
The probable issue is that glistmaker use 8 threads as default, but my computer have only 4 cores. this works:
glistmaker VL_0004.fasta -w 13 --num_threads 4 -o genome1
this is not working:
glistmaker VL_0004.fasta -w 13 -o genome1
How to change modeling.py to set glistmaker to use only 4 threads? I try to modify row 293, but with no success: 286 def get_kmer_lists(self): 287 # Makes "K-mer_lists" directory where all lists are stored. 288 # Generates k-mer lists for every sample in names_of_samples variable 289 # (list or dict). 290 run(["mkdir", "-p", "K-mer_lists"]) 291 process = run( 292 ["glistmaker " + self.address + " -o K-mer_lists/" + 293 self.name + "_0" + " -w " + self.kmer_length + " -c " + self.cutoff + " --num_threads 4 "], shell=True ) Input.lock.acquire() stderr_print.currentSampleNum.value += 1 Input.lock.release() stderr_print.print_progress("lists generated.")
Thank you for help, Jan
PS: Complete bash report:
~/PhenotypeSeeker/example/PS_modeling_example_analysis$ phenotypeseeker modeling PS_modeling_example_files/data.pheno
PhenotypeSeeker
modeling
Generating the k-mer lists for input samples: 30 of 30 lists generated. Generating the k-mer feature vector. Error: K-mer_lists/VL_0004_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0216_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0065_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0303_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0112_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0073_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0369_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0378_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0088_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0442_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0198_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0047_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0098_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0252_0_13.list is created with a newer glistmaker version. Error: K-mer_lists/VL_0302_0_13.list is created with a newer glistmaker version. gt4_wordmap_new: could not mmap file K-mer_lists/VL_0369_1_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0065_1_13_union.list Error: Creating the wordmap failed! gt4_wordmap_new: could not mmap file K-mer_lists/VL_0004_1_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0216_1_13_union.list Error: Creating the wordmap failed! gt4_wordmap_new: could not mmap file K-mer_lists/VL_0303_1_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0073_1_13_union.list Error: Creating the wordmap failed! gt4_wordmap_new: could not mmap file K-mer_lists/VL_0098_1_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0442_1_13_union.list Error: Creating the wordmap failed! gt4_wordmap_new: could not mmap file K-mer_lists/VL_0302_1_13_union.list Error: Cannot mmap K-mer_lists/VL_0302_1_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0198_1_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0088_1_13_union.list Error: Creating the wordmap failed! gt4_wordmap_new: could not mmap file K-mer_lists/VL_0378_1_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0112_1_13_union.list Error: Creating the wordmap failed! gt4_wordmap_new: could not mmap file K-mer_lists/VL_0004_2_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0378_2_13_union.list Error: Creating the wordmap failed! gt4_wordmap_new: could not mmap file K-mer_lists/VL_0369_2_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0303_2_13_union.list Error: Creating the wordmap failed! gt4_wordmap_new: could not mmap file K-mer_lists/VL_0098_2_13_union.list Error: Cannot mmap K-mer_lists/VL_0098_2_13_union.list gt4_wordmap_new: could not mmap file K-mer_lists/VL_0004_3_13_union.list Error: Cannot mmap K-mer_lists/VL_0004_3_13_union.list mv: cannot stat 'K-mer_lists/VL_0004_4_13_union.list': No such file or directory Mapping samples to the feature vector space: gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 3 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 4 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 7 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 8 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 9 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 12 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 13 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 15 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 16 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 22 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 23 of 30 samples mapped.gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted gt4_word_list_stream_new: could not open file K-mer_lists/feature_vector.list Error: K-mer_lists/feature_vector.list is invalid or corrupted 30 of 30 samples mapped. Deleting the existing distances.mat file... Deleting the existing reference.msh file... Deleting the existing mash_distances.mat file... Estimating the Mash distances between samples... Calculating the GSC weights from mash distance matrix... Conducting the k-mer specific chi-square tests: Cannot open list K-mer_lists/feature_vector.list rm: cannot remove 'K-mer_lists/feature_vector.list': No such file or directory ^CTraceback (most recent call last): File "/usr/local/bin/phenotypeseeker", line 313, in
Main()
File "/usr/local/bin/phenotypeseeker", line 302, in Main
func = args.func(args)
File "/home/honza/.local/lib/python3.8/site-packages/PhenotypeSeeker/modeling.py", line 1681, in modeling
list(map(
File "/home/honza/.local/lib/python3.8/site-packages/PhenotypeSeeker/modeling.py", line 1682, in
lambda x: x.test_kmers_association_with_phenotype(),
File "/home/honza/.local/lib/python3.8/site-packages/PhenotypeSeeker/modeling.py", line 618, in test_kmers_association_with_phenotype
pvalues_from_all_threads = Input.pool.map(
File "/usr/local/lib/python3.8/dist-packages/multiprocess/pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/local/lib/python3.8/dist-packages/multiprocess/pool.py", line 762, in get
self.wait(timeout)
File "/usr/local/lib/python3.8/dist-packages/multiprocess/pool.py", line 759, in wait
self._event.wait(timeout)
File "/usr/lib/python3.8/threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
File "/usr/lib/python3.8/threading.py", line 302, in wait
waiter.acquire()
KeyboardInterrupt