Closed LliliansCalvo closed 6 years ago
Hi, did you by any chance forget a ">" character in your Fasta files?
Kevin
From: LliliansCalvo notifications@github.com Sent: Wednesday, November 22, 2017 9:30:06 AM To: ldiao/MixMir Cc: Subscribed Subject: [ldiao/MixMir] Mixmir error (#3)
Hi, I am trying to run mirmix on a list, rather small, of 3'UTR from the gene eve in different insects species. However I keep getting the same error and I cannot figure out what's the problem.
My fasta file looks like :
apis_mellifera_eve GAA.... bombyx_mori_eve TTTAGTT.. drosophila_melanogaster_eve GCCCGCGA.. (total of 9 different sequences in FASTA format)
My ID ranked files looks like (rank is arbitrary and was created by myself) : apis_mellifera_eve 3 bombyx_mori_eve -2 drosophila_melanogaster_eve 4 drosophila_sechillia_eve 4 drosophila_simulans_eve 4 drosophila_yakuba_eve 4 oncopeltus_fasciatus_eve -3 nasonia_vitripennis_eve -1 tribolium_castaneum_eve -2 (total of 9 different UTRs)
I am using microRNA mature sequences downloaded from mirbase:
dps-miR-2517a-3p UAUUG.. dps-miR-2543a-5p AUCCUAU.. dps-miR-2543a-3p UAUGCA.. dps-miR-2543b-5p AUCCUA..
my command is : MixMir-master llilians$ python MixMir.py --seqf all_utr3p.fa --exprf IDs_ranked.txt --mirf arthropods-mature.fa --k_kin 6 --k_motif 6 --N 20 --fast 0 --out results
Error : Solving MLM with GEMMA Parsing files for PLINK Traceback (most recent call last): File "MixMir.py", line 157, in doAll(doKin=doKin) File "MixMir.py", line 18, in doAll runParse(doKin=doKin) File "MixMir.py", line 43, in runParse parseAll.doAll(doKin=doKin,seqf=seqf,exprf=exprf,outfnkin=kinf,outPedFile=outPedFile,outMapFile=outMapFile,kkin=kkin,kmotif=kmotif,frac=frac,useFast=useFast) File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 46, in doAll exprs = loadmic(fname=exprf) File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 112, in loadmic exprs = [[row[0],float(row[1])] for row in exprs] File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 112, in exprs = [[row[0],float(row[1])] for row in exprs] IndexError: list index out of range
Could you please help me to figure out what's the problem here ? Thanks,
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fldiao%2FMixMir%2Fissues%2F3&data=02%7C01%7Ckcchen%40dls.rutgers.edu%7C1f9f617e731742b2a27108d531b588bd%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636469578115980164&sdata=lt3VfS4R4v2oWc%2Fvzi8ypzOML7nq%2FEpYEwtbJ4Z5LvE%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGXb-ro-XOiCcDyycdOD5ULqaQJpOL0uks5s5C_ugaJpZM4QnfE1&data=02%7C01%7Ckcchen%40dls.rutgers.edu%7C1f9f617e731742b2a27108d531b588bd%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636469578115980164&sdata=fB4XA1eprPQrPEeVjh%2B2%2BgxUuzu8DivijFxM1bkmdCo%3D&reserved=0.
Hi, The FASTA file looks fine with the '>' Here I add a screenshot of my FASTA file
The mirbase FASTA file also looks like that containing only the mature sequences. Thanks
Also when I try to run the example from the README file, it doesn't work neither and I get an error message like this :
l-u-c02sl2h9gg78:MixMir-master llilians$ python MixMir.py --seqf testdat/test-utrs.fa --exprf testdat/test-exprs.txt --mirf testdat/testmirs.fa --k_kin 6 --k_motif 6 --N 20 --fast 0 --out testdat/test
Solving MLM with GEMMA
Parsing files for PLINK
Removing 418 motifs with no information
Removing 418 motifs with no information
Traceback (most recent call last):
File "MixMir.py", line 157, in
Hi LliliansCalvo,
I've just returned from the Thanksgiving holiday and can try to take a look at this issue tonight after I get home from work.
Thanks, Liyang
On Wed, Nov 29, 2017 at 6:52 AM, LliliansCalvo notifications@github.com wrote:
Also when I try to run the example from the README file, it doesn't work neither and I get an error message like this :
l-u-c02sl2h9gg78:MixMir-master llilians$ python MixMir.py --seqf testdat/test-utrs.fa --exprf testdat/test-exprs.txt --mirf testdat/testmirs.fa --k_kin 6 --k_motif 6 --N 20 --fast 0 --out testdat/test Solving MLM with GEMMA Parsing files for PLINK Removing 418 motifs with no information Removing 418 motifs with no information Traceback (most recent call last): File "MixMir.py", line 157, in doAll(doKin=doKin) File "MixMir.py", line 18, in doAll runParse(doKin=doKin) File "MixMir.py", line 43, in runParse parseAll.doAll(doKin=doKin,seqf=seqf,exprf=exprf,outfnkin=kinf,outPedFile= outPedFile,outMapFile=outMapFile,kkin=kkin,kmotif= kmotif,frac=frac,useFast=useFast) File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 76, in doAll makePed(genes=genes,dcounts=dcounts,dexprs=dexprs,outfn=outPedFile) File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 281, in makePed newrow = [g,dexprs[g]] + map(isZero,dcounts[g]) TypeError: can only concatenate list (not "map") to list
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ldiao/MixMir/issues/3#issuecomment-347837981, or mute the thread https://github.com/notifications/unsubscribe-auth/AGbgBr-kbZbOONZv6oqJAGyEDopDhVfAks5s7UWLgaJpZM4QnfE1 .
Hi, this sounds like it might be a Python 2 vs. Python 3 issue to me?
Kevin
From: ldiao notifications@github.com Sent: Wednesday, November 29, 2017 9:27:06 AM To: ldiao/MixMir Cc: Kevin Chen; Comment Subject: Re: [ldiao/MixMir] Mixmir error (#3)
Hi LliliansCalvo,
I've just returned from the Thanksgiving holiday and can try to take a look at this issue tonight after I get home from work.
Thanks, Liyang
On Wed, Nov 29, 2017 at 6:52 AM, LliliansCalvo notifications@github.com wrote:
Also when I try to run the example from the README file, it doesn't work neither and I get an error message like this :
l-u-c02sl2h9gg78:MixMir-master llilians$ python MixMir.py --seqf testdat/test-utrs.fa --exprf testdat/test-exprs.txt --mirf testdat/testmirs.fa --k_kin 6 --k_motif 6 --N 20 --fast 0 --out testdat/test Solving MLM with GEMMA Parsing files for PLINK Removing 418 motifs with no information Removing 418 motifs with no information Traceback (most recent call last): File "MixMir.py", line 157, in doAll(doKin=doKin) File "MixMir.py", line 18, in doAll runParse(doKin=doKin) File "MixMir.py", line 43, in runParse parseAll.doAll(doKin=doKin,seqf=seqf,exprf=exprf,outfnkin=kinf,outPedFile= outPedFile,outMapFile=outMapFile,kkin=kkin,kmotif= kmotif,frac=frac,useFast=useFast) File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 76, in doAll makePed(genes=genes,dcounts=dcounts,dexprs=dexprs,outfn=outPedFile) File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 281, in makePed newrow = [g,dexprs[g]] + map(isZero,dcounts[g]) TypeError: can only concatenate list (not "map") to list
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ldiao/MixMir/issues/3#issuecomment-347837981, or mute the thread https://github.com/notifications/unsubscribe-auth/AGbgBr-kbZbOONZv6oqJAGyEDopDhVfAks5s7UWLgaJpZM4QnfE1 .
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fldiao%2FMixMir%2Fissues%2F3%23issuecomment-347875915&data=02%7C01%7Ckcchen%40dls.rutgers.edu%7Cec3ead8560f447c46bd108d53735453b%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636475624289085786&sdata=P6IrIqxqCfSkXtXye7AlXHJlngjI5ySZAbt3UJe9C0Q%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGXb-gbC0Jz5yD1gw7y--cPxotDfdY-rks5s7Wm6gaJpZM4QnfE1&data=02%7C01%7Ckcchen%40dls.rutgers.edu%7Cec3ead8560f447c46bd108d53735453b%7Cb92d2b234d35447093ff69aca6632ffe%7C1%7C0%7C636475624289085786&sdata=6wcuAC9ZHwDaNYxZM0h17IxzvetgB6bKonSF3cn2YQQ%3D&reserved=0.
Thanks Kevin,
I repeat it all using a different computer (with Python version 2.7) However, I am still getting an error, a different one this time.
Toms-iMac:MixMir-master tompettini$ python MixMir.py --seqf testdat/test-utrs.fa --exprf testdat/test-exprs.txt --mirf testdat/testmirs.fa --k_kin 6 --k_motif 6 --N 20 --fast 0 --out testdat/test
Solving MLM with GEMMA
Parsing files for PLINK
Removing 418 motifs with no information
Removing 418 motifs with no information
Finished parsing files!
Running PLINK...........................
sh: ./plink: No such file or directory
PLINK files generated!
Solving MLM.................
sh: ./gemma-0.94: cannot execute binary file
sh: ./gemma-0.94: cannot execute binary file
MLM solve complete!
Analyzing results.......................
Traceback (most recent call last):
File "MixMir.py", line 157, in
This command produced 3 different files :
Thanks again, Llilians
Hi Llilians,
I was able to run the example provided without an issue, using the command you included above. Based on the error messages, did you make sure to chmod +x gemma-0.94?
On Wed, Nov 29, 2017 at 12:10 PM, LliliansCalvo notifications@github.com wrote:
Thanks Kevin,
I repeat it all using a different computer (with Python version 2.7) However, I am still getting an error, a different one this time.
Toms-iMac:MixMir-master tompettini$ python MixMir.py --seqf testdat/test-utrs.fa --exprf testdat/test-exprs.txt --mirf testdat/testmirs.fa --k_kin 6 --k_motif 6 --N 20 --fast 0 --out testdat/test Solving MLM with GEMMA Parsing files for PLINK Removing 418 motifs with no information Removing 418 motifs with no information Finished parsing files! Running PLINK........................... sh: ./plink: No such file or directory PLINK files generated! Solving MLM................. sh: ./gemma-0.94: cannot execute binary file sh: ./gemma-0.94: cannot execute binary file MLM solve complete! Analyzing results....................... Traceback (most recent call last): File "MixMir.py", line 157, in doAll(doKin=doKin) File "MixMir.py", line 31, in doAll getResults() File "MixMir.py", line 89, in getResults analyze.doAll(resFile=resFile,mirFile=mirf,seqFile=seqf,N= Nkeep,useFast=useFast,outfn=resf) File "/Users/tompettini/Desktop/MixMir-master/analyze.py", line 34, in doAll res = loadGemmaRes(f=resFile,reverse=True) File "/Users/tompettini/Desktop/MixMir-master/analyze.py", line 141, in loadGemmaRes gem = read(f) File "/Users/tompettini/Desktop/MixMir-master/analyze.py", line 54, in read ifile = open(f,'r') IOError: [Errno 2] No such file or directory: 'output/test.assoc.txt'
This command produced 3 different files :
- kin.txt
- .ped
- .map
Thanks again, Llilians
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ldiao/MixMir/issues/3#issuecomment-347929201, or mute the thread https://github.com/notifications/unsubscribe-auth/AGbgBqzyg7zeZ9PYkVA0pOLqZ3mflIiPks5s7ZAMgaJpZM4QnfE1 .
Hi Idiao,
I did chmod +x gemma-0.94 in both computers I tried.
Thanks !
Did you put a symlink of the plink executable in the same directory as MixMir.py? You can test to make sure plink is working first by running
./plink --noweb
This should bring up the PLINK welcome/startup message (see below).
Here is an example of output I get when running the example command, which shows PLINK starting up and running first.
@----------------------------------------------------------@ | PLINK! | v1.07 | 10/Aug/2009 |
---|---|---|---|
(C) 2009 Shaun Purcell, GNU General Public License, v2 | |||
---------------------------------------------------------- | |||
For documentation, citation & bug-report instructions: | |||
http://pngu.mgh.harvard.edu/purcell/plink/ |
@----------------------------------------------------------@
Skipping web check... [ --noweb ] Writing this text to log file [ testdat/test.log ] Analysis started: Wed Nov 29 19:20:47 2017
Options in effect: --noweb --make-bed --file testdat/test --no-fid --no-parents --no-sex --map3 --out testdat/test
3678 (of 3678) markers to be included from [ testdat/test.map ] Warning, found 24 individuals with ambiguous sex codes Writing list of these individuals to [ testdat/test.nosex ] 24 individuals read from [ testdat/test.ped ] 24 individuals with nonmissing phenotypes Assuming a quantitative trait Missing phenotype value is -9 0 males, 0 females, and 24 of unspecified sex Before frequency and genotyping pruning, there are 3678 SNPs 24 founders and 0 non-founders found Total genotyping rate in remaining individuals is 1 0 SNPs failed missingness test ( GENO > 1 ) 0 SNPs failed frequency test ( MAF < 0 ) After frequency and genotyping pruning, there are 3678 SNPs After filtering, 24 individuals with non-missing status After filtering, 0 males, 0 females, and 24 of unspecified sex Writing pedigree information to [ testdat/test.fam ] Writing map (extended format) information to [ testdat/test.bim ] Writing genotype bitfile to [ testdat/test.bed ] Using (default) SNP-major mode
Analysis finished: Wed Nov 29 19:20:47 2017
Reading Files ...
Start Eigen-Decomposition... Reading Files ...
pve estimate =0.999988 se(pve) =9.6319e-06 Reading SNPs 0.00%^MReading SNPs ==================================================100.00% Solving MLM with GEMMA Parsing files for PLINK Removing 418 motifs with no information Removing 418 motifs with no information Finished parsing files! Running PLINK........................... PLINK files generated! Solving MLM................. MLM solve complete! Analyzing results....................... (12, 'of', 20, 'entries matched') Writing MixMir results to testdat/test-MixMir-results.txt.gemma Analysis complete!
From: LliliansCalvo Sent: Wednesday, November 29, 2017 7:13 PM To: ldiao/MixMir Cc: ldiao; Comment Subject: Re: [ldiao/MixMir] Mixmir error (#3)
Hi Idiao, I did chmod +x gemma-0.94 in both computers I tried. Thanks ! — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
I can reproduce your error if there is no plink in the working directory, so I suspect this is the error:
(venv27) bash-4.2$ python MixMir.py --seqf testdat/test-utrs.fa --exprf testdat/test-exprs.txt --mirf testdat/testmirs.fa --k_kin 6 --k_motif 6 --N 20 --fast 0 --out testdat/test_noplink
Solving MLM with GEMMA
Parsing files for PLINK
Removing 418 motifs with no information
Removing 418 motifs with no information
Finished parsing files!
Running PLINK...........................
sh: ./plink: No such file or directory
PLINK files generated!
Solving MLM.................
error! fail to open .bim file: testdat/test_noplink.bim
error! fail to open .bed file: testdat/test_noplink.bed
error! fail to open .fam file: testdat/test_noplink.fam
error! fail to open .bim file: testdat/test_noplink.bim
error! fail to open .bed file: testdat/test_noplink.bed
error! fail to open .fam file: testdat/test_noplink.fam
MLM solve complete!
Analyzing results.......................
Traceback (most recent call last):
File "MixMir.py", line 157, in
From: LliliansCalvo Sent: Wednesday, November 29, 2017 7:13 PM To: ldiao/MixMir Cc: ldiao; Comment Subject: Re: [ldiao/MixMir] Mixmir error (#3)
Hi Idiao, I did chmod +x gemma-0.94 in both computers I tried. Thanks ! — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
Thanks,
Maybe you are right and is Plink the error, but I have no idea how to fix it. It seems like something is missing
l-u-c02sl2h9gg78:MixMir-master llilians$ ./plink --noweb
@----------------------------------------------------------@ | PLINK! | v1.07 | 10/Aug/2009 |
---|---|---|---|
(C) 2009 Shaun Purcell, GNU General Public License, v2 | |||
---------------------------------------------------------- | |||
For documentation, citation & bug-report instructions: | |||
http://pngu.mgh.harvard.edu/purcell/plink/ |
@----------------------------------------------------------@
Skipping web check... [ --noweb ] Writing this text to log file [ plink.log ] Analysis started: Thu Nov 30 10:42:15 2017
Options in effect: --noweb
Before frequency and genotyping pruning, there are 0 SNPs 0 founders and 0 non-founders found 0 SNPs failed missingness test ( GENO > 1 ) 0 SNPs failed frequency test ( MAF < 0 ) After frequency and genotyping pruning, there are 0 SNPs
ERROR: Stopping as there are no SNPs left for analysis
Working on another computer (Linux) the problem has been solved. Your example runs smoothly 👍 However, when I run my own file the error persist
python MixMir.py --seqf 13_brief-names_utr3p.fa --exprf test-exprs-copy.txt --mirf arthropods-mature.fa --k_kin 6 --k_motif 6 --N 50 --fast 0 --out results-13-utr3p
Solving MLM with GEMMA
Parsing files for PLINK
Traceback (most recent call last):
File "MixMir.py", line 157, in
This is how my files look like, I have modified them to look more like yours:
fasta utr3p
ame_refGene_NM_001008533 strand=+ GAACGA... bmo_refGene_NM_001029878 strand=+ TTTAGTTTTCC... dme_refGene_NM_001039510 strand=+ GCCCGCGATCC... (13 UTRs in total) (For some reasons when I post this the '>' will disappear from the post, but they are in the file (ex: >ame_refGene_NM_001008533 strand=+ GAACGA...)
ID file NM_001008533 -0.01531657 NM_001039510 -0.12028854 NM_001160371 -0.01531657
microRNAs mature sequences (fro mirbase)
aae-miR-184 UGGACGGAGAACUGAUAAGGGC aae-miR-275-5p CGCGCUAAGCAGGAACCGAGAC etc
Sorry for bothering, Thanks !
Can you try replacing U with T in your microRNAs file to see if that works? Also to reiterate Kevin's comment previously, make sure to have the > in the fasta files (including miRNA file).
On Thu, Nov 30, 2017 at 8:23 AM, Llili notifications@github.com wrote:
Working on another computer (Linux) the problem has been solved. Your example runs smoothly 👍 However, when I run my own file the error persist
python MixMir.py --seqf 13_brief-names_utr3p.fa --exprf test-exprs-copy.txt --mirf arthropods-mature.fa --k_kin 6 --k_motif 6 --N 50 --fast 0 --out results-13-utr3p
Solving MLM with GEMMA Parsing files for PLINK Traceback (most recent call last): File "MixMir.py", line 157, in doAll(doKin=doKin) File "MixMir.py", line 18, in doAll runParse(doKin=doKin) File "MixMir.py", line 43, in runParse parseAll.doAll(doKin=doKin,seqf=seqf,exprf=exprf,outfnkin=kinf,outPedFile= outPedFile,outMapFile=outMapFile,kkin=kkin,kmotif= kmotif,frac=frac,useFast=useFast) File "/home/ana/mixmir/MixMir-master/parseAll.py", line 49, in doAll dutrs = makeFaDict(utrs) File "/home/ana/mixmir/MixMir-master/parseAll.py", line 125, in makeFaDict if row[0] == '>': IndexError: string index out of range
This is how my files look like, I have modified them to look more like yours:
- fasta utr3p
ame_refGene_NM_001008533 strand=+ GAACGA... bmo_refGene_NM_001029878 strand=+ TTTAGTTTTCC... dme_refGene_NM_001039510 strand=+ GCCCGCGATCC... (13 UTRs in total)
-
ID file NM_001008533 -0.01531657 NM_001039510 -0.12028854 NM_001160371 -0.01531657
microRNAs mature sequences (fro mirbase)
aae-miR-184 UGGACGGAGAACUGAUAAGGGC aae-miR-275-5p CGCGCUAAGCAGGAACCGAGAC etc
Sorry for bothering, Thanks !
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ldiao/MixMir/issues/3#issuecomment-348186626, or mute the thread https://github.com/notifications/unsubscribe-auth/AGbgBlTaqHD3h-obTkGXspNk-X8XnMY9ks5s7qxlgaJpZM4QnfE1 .
Thanks again for the help !
MixMir now working, moreover thanks for the advice Idiao, the miR file does need to have T instead of U.
Hi, I am trying to run mirmix on a list, rather small, of 3'UTR from the gene eve in different insects species. However I keep getting the same error and I cannot figure out what's the problem.
My fasta file looks like :
My ID ranked files looks like (rank is arbitrary and was created by myself) : apis_mellifera_eve 3 bombyx_mori_eve -2 drosophila_melanogaster_eve 4 drosophila_sechillia_eve 4 drosophila_simulans_eve 4 drosophila_yakuba_eve 4 oncopeltus_fasciatus_eve -3 nasonia_vitripennis_eve -1 tribolium_castaneum_eve -2 (total of 9 different UTRs)
I am using microRNA mature sequences downloaded from mirbase:
my command is : MixMir-master llilians$ python MixMir.py --seqf all_utr3p.fa --exprf IDs_ranked.txt --mirf arthropods-mature.fa --k_kin 6 --k_motif 6 --N 20 --fast 0 --out results
Error : Solving MLM with GEMMA Parsing files for PLINK Traceback (most recent call last): File "MixMir.py", line 157, in
doAll(doKin=doKin)
File "MixMir.py", line 18, in doAll
runParse(doKin=doKin)
File "MixMir.py", line 43, in runParse
parseAll.doAll(doKin=doKin,seqf=seqf,exprf=exprf,outfnkin=kinf,outPedFile=outPedFile,outMapFile=outMapFile,kkin=kkin,kmotif=kmotif,frac=frac,useFast=useFast)
File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 46, in doAll
exprs = loadmic(fname=exprf)
File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 112, in loadmic
exprs = [[row[0],float(row[1])] for row in exprs]
File "/Users/llilians/Desktop/MixMir-master/parseAll.py", line 112, in
exprs = [[row[0],float(row[1])] for row in exprs]
IndexError: list index out of range
Could you please help me to figure out what's the problem here ? Thanks,