Closed antonvila closed 5 years ago
Hi Anton,
could you try running it with the -D
flag, which gives a more verbose output of errors!
From the output, 5 clusters seem quite few. Are you trying to run price on a subset (mitochondrial only, as suggested by filenames) of all reads? I would advice against that, as to build an accurate codon model, price needs many reads!
Best, Florian
Here is the output:
anton@anton-H87M-D3H:/media/anton/Maxtor/AVS/DATA/PRICE_032718/mt-genome$ gedi -e Price -reads /media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam -genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress -D 2018-04-03 11:35:55.053 INFO Command: gedi -e Price -reads /media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam -genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress -D 2018-04-03 11:35:55.210 INFO Discovering classes in classpath 2018-04-03 11:35:55.313 INFO Preparing simple class references 2018-04-03 11:35:55.485 INFO Gedi 1.0.2 (JAR) startup 2018-04-03 11:35:56.366 INFO Reading oml /home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml 2018-04-03 11:35:56.381 INFO Done reading oml /home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml PRICE PRICE is an analysis method for Ribo-seq data.
Error: java.lang.RuntimeException: Could not run gedi.riboseq.javapipeline.PriceEstimateModel@62c32db Could not run gedi.riboseq.javapipeline.PriceEstimateModel@62c32db 0
gedi -e PRICE
General:
-reads
Commandline: -progress Show progress -D Verbose output of errors -h Show usage -hh Show verbose usage -hhh Show extra verbose usage -dry Dry run -keep Do not remove temp files
java.lang.RuntimeException: java.lang.RuntimeException: Could not run gedi.riboseq.javapipeline.PriceEstimateModel@62c32db at gedi.util.job.schedule.DefaultPetriNetScheduler.run(DefaultPetriNetScheduler.java:222) at gedi.util.program.GediProgram$1.execute(GediProgram.java:359) at gedi.util.program.GediProgram.run(GediProgram.java:204) at executables.Price.main(Price.java:68) Caused by: java.lang.RuntimeException: Could not run gedi.riboseq.javapipeline.PriceEstimateModel@62c32db at gedi.util.program.GediProgramJob.execute(GediProgramJob.java:63) at gedi.util.program.GediProgramJob.execute(GediProgramJob.java:27) at gedi.util.job.FireTransition.call(FireTransition.java:54) at gedi.util.job.FireTransition.call(FireTransition.java:25) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ArrayIndexOutOfBoundsException: 0 at gedi.util.ArrayUtils.argmax(ArrayUtils.java:1338) at gedi.riboseq.javapipeline.PriceEstimateModel.execute(PriceEstimateModel.java:87) at gedi.util.program.GediProgramJob.execute(GediProgramJob.java:61) ... 7 more
Dear Anton, this has to do with an inherent problem of mitochondrial transcripts. They are extremely densely packed, and PRICE needs reads with annotated start codons in their P site to fix the probability distribution of its codon model. So, if you really want to do this for mitochondrial reads, you face the problem that PRICE is not able to find many reads where it can be sure that it is one with an annotated start codon in the P site. (The ArrayIndexOutOfBoundsException tells me that in fact it found none!)
What you could do is to manually create the result file of this step and restart price: Create a file ${prefix}.maxPos
(where ${prefix} is your -prefix
parameter) with the following content:
Position 0
-12 1000
Use tabs as separators; the meaning of this is that you pretend that this step of PRICE has found 1000 reads at position -12 before a start codon (thus, PRICE will try to move the mode of the codon distribution to -12)
Best, Florian
Hi Florian,
thanks for the help! When I run PRICE with the fake ${prefix}.maxPos result file, I get the following error.
anton@anton-H87M-D3H:/media/anton/Maxtor/AVS/DATA/PRICE_032718/mt-genome$
gedi -e Price -reads
/media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam
-genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress
-D2018-04-05 20:03:34.816 INFO Command: gedi -e Price -reads
/media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam
-genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress
-D2018-04-05 20:03:34.920 INFO Discovering classes in classpath2018-04-05
20:03:34.996 INFO Preparing simple class references2018-04-05 20:03:35.101
INFO Gedi 1.0.2 (JAR) startup2018-04-05 20:03:35.202 INFO Reading oml
/home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml2018-04-05 20:03:35.207
INFO Done reading oml
/home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.omlPRICEPRICE is an
analysis method for Ribo-seq data.Error:java.lang.RuntimeException: Could
not run gedi.riboseq.javapipeline.PriceEstimateModel@2a2b6d58Could not run
gedi.riboseq.javapipeline.PriceEstimateModel@2a2b6d58gedi.util.datastructure.dataframe.IntegerDataColumn
cannot be cast to gedi.util.datastructure.dataframe.DoubleDataColumngedi -e
PRICE
On Wed, Apr 4, 2018 at 9:15 AM, florianerhard notifications@github.com wrote:
Dear Anton, this has to do with an inherent problem of mitochondrial transcripts. They are extremely densely packed, and PRICE needs reads with annotated start codons in their P site to fix the probability distribution of its codon model. So, if you really want to do this for mitochondrial reads, you face the problem that PRICE is not able to find many reads where it can be sure that it is one with an annotated start codon in the P site. (The ArrayIndexOutOfBoundsException tells me that in fact it found none!)
What you could do is to manually create the result file of this step and restart price: Create a file ${prefix}.maxPos (where ${prefix} is your -prefix parameter) with the following content:
Position 0 -12 1000
Use tabs as separators; the meaning of this is that you pretend that this step of PRICE has found 1000 reads at position -12 before a start codon (thus, PRICE will try to move the mode of the codon distribution to -12)
Best, Florian
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/erhard-lab/price/issues/7#issuecomment-378503913, or mute the thread https://github.com/notifications/unsubscribe-auth/AG4fRNDCikdyF9sIMGrPpfn3ZcMM4y8Rks5tlHL6gaJpZM4TDc2K .
-- Antón Vila-Sanjurjo, PhD Project PI Departamento de Bioloxía Celular e Molecular Facultade de Ciencias Universidade de A Coruña (UDC) Campus Zapateira, s/n 15.071 - A Coruña - España (Spain).
tlf: (34) 981-167000 ext:2659 e-mail: antonvila.s@gmail.com
Oh, dammit... that's my fault. It has to be
Position 0
-12 1000.0
Hi Florian,
it gets a little further and then it dies:
anton@anton-H87M-D3H:/media/anton/Maxtor/AVS/DATA/PRICE_032718/mt-genome$
gedi -e Price -reads
/media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam
-genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress
-D2018-04-06 11:56:13.049 INFO Command: gedi -e Price -reads
/media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam
-genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress
-D2018-04-06 11:56:13.232 INFO Discovering classes in classpath2018-04-06
11:56:13.474 INFO Preparing simple class references2018-04-06 11:56:13.621
INFO Gedi 1.0.2 (JAR) startup2018-04-06 11:56:14.688 INFO Reading oml
/home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml2018-04-06 11:56:14.704
INFO Done reading oml
/home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml2018-04-06 11:56:15.345
INFO Using maxpos=12 (Merged)2018-04-06 11:56:15.346 INFO Estimate
parametersProcessed 17366795 elements in 2m 52s 162ms (Throughput:
100874.1/sec)Correcting from 16 to 13 (desired: 12)2018-04-06 11:59:07.613
INFO LL=-2825.99PRICEPRICE is an analysis method for Ribo-seq
data.Error:java.lang.RuntimeException: Could not run
gedi.riboseq.javapipeline.PriceCodonInference@69e9cf6Could not run
gedi.riboseq.javapipeline.PriceCodonInference@69e9cf6Could not load
storage!Unexpected char 103 at (line no=1, column no=1, offset=0)gedi -e
PRICE
On Fri, Apr 6, 2018 at 11:35 AM, florianerhard notifications@github.com wrote:
Oh, dammit... that's my fault. It has to be
Position 0 -12 1000.0
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/erhard-lab/price/issues/7#issuecomment-379201759, or mute the thread https://github.com/notifications/unsubscribe-auth/AG4fRPvhrZ8U9v0NvBur7WgzYfERhkWlks5tlzbzgaJpZM4TDc2K .
-- Antón Vila-Sanjurjo, PhD Project PI Departamento de Bioloxía Celular e Molecular Facultade de Ciencias Universidade de A Coruña (UDC) Campus Zapateira, s/n 15.071 - A Coruña - España (Spain).
tlf: (34) 981-167000 ext:2659 e-mail: antonvila.s@gmail.com
Dear Anton,
this seems to be a problem with the ...clusters.cit
file from the previous run (with the initial exception). I guess it was not written to disk completely before Price crashed.
Could you delete the old output files of Price from the run where you had the initial error?
Best, Florian
Hi Florian,
I've been very busy for the last few weeks, so I couldn't try your April-9 solution. I have now and got the following result:
*anton@anton-H87M-D3H:/media/anton/Maxtor/AVS/DATA/PRICE_032718/mt-genome$ gedi -e Price -reads /media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam -genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress -D2018-05-08 13:11:34.468 INFO Command: gedi -e Price -reads /media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam -genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress -D2018-05-08 13:11:34.585 INFO Discovering classes in classpath2018-05-08 13:11:34.666 INFO Preparing simple class references2018-05-08 13:11:34.789 INFO Gedi 1.0.2 (JAR) startup2018-05-08 13:11:34.892 INFO Reading oml /home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml2018-05-08 13:11:34.896 INFO Done reading oml /home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml2018-05-08 13:11:34.968 INFO Estimating maxpos...2018-05-08 13:11:34.974 INFO Clustering readsProcessed 36701 elements in 13s 718ms (Throughput: 2675.4/sec)Processed 36701 elements in 13s 935ms (Throughput: 2633.7/sec)Processed 36701 elements in 13s 851ms (Throughput: 2649.7/sec)2018-05-08 13:11:48.971 INFO Found 230 clusters without looking at annotation2018-05-08 13:11:49.040 INFO Found 9 clusters after looking at annotation2018-05-08 13:11:49.043 INFO Found 5 clusters after filtering with read/region countPRICEPRICE is an analysis method for Ribo-seq data.Error:java.lang.RuntimeException: Could not run gedi.riboseq.javapipeline.PriceEstimateModel@10c87727Could not run gedi.riboseq.javapipeline.PriceEstimateModel@10c877270gedi -e PRICE
Dear Anton,
I tried to reproduce your error. The only way such an error could occur is when there are no positions <=10 in the file you supplied manually. Have you forgotten die -
in front of -12
?
Florian
Hi Florian,
you were right, it was a problem with the file added manually. Tried again and, after a while, I got a new error.
anton@anton-H87M-D3H:/media/anton/Maxtor/AVS/DATA/PRICE_032718/mt-genome$ gedi -e Price -reads /media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam -genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress -D 2018-05-15 10:08:15.328 INFO Command: gedi -e Price -reads /media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam -genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 -progress -D 2018-05-15 10:08:15.463 INFO Discovering classes in classpath 2018-05-15 10:08:15.553 INFO Preparing simple class references 2018-05-15 10:08:15.667 INFO Gedi 1.0.2 (JAR) startup 2018-05-15 10:08:15.838 INFO Reading oml /home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml 2018-05-15 10:08:15.862 INFO Done reading oml /home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml 2018-05-15 10:08:16.038 INFO Clustering reads
Error: java.lang.RuntimeException: Could not run gedi.riboseq.javapipeline.PriceOrfInference@4325e028 Could not run gedi.riboseq.javapipeline.PriceOrfInference@4325e028 Exception in parallel iterator thread! 0
gedi -e PRICE
General:
-reads
Commandline: -progress Show progress -D Verbose output of errors -h Show usage -hh Show verbose usage -hhh Show extra verbose usage -dry Dry run -keep Do not remove temp files
java.lang.RuntimeException: java.lang.RuntimeException: Could not run gedi.riboseq.javapipeline.PriceOrfInference@4325e028 at gedi.util.job.schedule.DefaultPetriNetScheduler.run(DefaultPetriNetScheduler.java:222) at gedi.util.program.GediProgram$1.execute(GediProgram.java:359) at gedi.util.program.GediProgram.run(GediProgram.java:204) at executables.Price.main(Price.java:68) Caused by: java.lang.RuntimeException: Could not run gedi.riboseq.javapipeline.PriceOrfInference@4325e028 at gedi.util.program.GediProgramJob.execute(GediProgramJob.java:63) at gedi.util.program.GediProgramJob.execute(GediProgramJob.java:27) at gedi.util.job.FireTransition.call(FireTransition.java:54) at gedi.util.job.FireTransition.call(FireTransition.java:25) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Exception in parallel iterator thread! at gedi.util.functions.ParallelizedIterator.fatal(ParallelizedIterator.java:221) at gedi.util.functions.ParallelizedIterator.tryNext(ParallelizedIterator.java:189) at gedi.util.functions.ParallelizedIterator.hasNext(ParallelizedIterator.java:150) at java.util.Iterator.forEachRemaining(Iterator.java:115) at gedi.riboseq.utils.RiboUtils.lambda$processCodonsSink$4(RiboUtils.java:106) at gedi.riboseq.utils.RiboUtils.processCodons(RiboUtils.java:146) at gedi.riboseq.utils.RiboUtils.processCodonsSink(RiboUtils.java:106) at gedi.riboseq.javapipeline.PriceOrfInference.execute(PriceOrfInference.java:118) at gedi.util.program.GediProgramJob.execute(GediProgramJob.java:61) ... 7 more Caused by: java.lang.ArrayIndexOutOfBoundsException: 0 at jdistlib.math.spline.SmoothSpline.findDuplicateIndices(SmoothSpline.java:1508) at jdistlib.math.spline.SmoothSpline.fit(SmoothSpline.java:1392) at jdistlib.math.spline.SmoothSpline.fitDFMatch(SmoothSpline.java:1296) at gedi.riboseq.inference.orf.NoiseModel.computeMeanSpline(NoiseModel.java:188) at gedi.riboseq.inference.orf.NoiseModel.getProbability(NoiseModel.java:141) at gedi.riboseq.inference.orf.NoiseModel.getProbability0(NoiseModel.java:121) at gedi.riboseq.inference.orf.OrfInference.inFrameTestCandidate(OrfInference.java:1271) at gedi.riboseq.inference.orf.OrfInference.inFrameTest(OrfInference.java:1079) at gedi.riboseq.inference.orf.OrfInference.inferOrfs(OrfInference.java:433) at gedi.riboseq.javapipeline.PriceOrfInference.lambda$null$1(PriceOrfInference.java:119) at gedi.util.FunctorUtils$DemultiplexIterator.lookAhead(FunctorUtils.java:106) at gedi.util.FunctorUtils$DemultiplexIterator.hasNext(FunctorUtils.java:90) at gedi.util.FunctorUtils$PeekIterator.hasNext(FunctorUtils.java:636) at gedi.util.FunctorUtils$SideEffectIterator.hasNext(FunctorUtils.java:717) at gedi.util.functions.ExtendedIterator.toCollection(ExtendedIterator.java:375) at gedi.util.functions.ParallelizedIterator$Worker.run(ParallelizedIterator.java:333)
Dear Anton,
I guess this means that doing this for the mitochondrial genome only is not possible. In order to compute the p values for ORF identification, we need a way to judge the noise levels of the identified codons. In PRICE this is done by fitting splines to the graph of (translation strength)-(noise level) in annotated ORFs. If you have to few, this might not work and this is what the error means.
You could do a run on the full data, and then copy the ${prefix}.noise.model of this run into the ${prefix}.noise.model of the mito-run. This is ok only if you assume that the noise levels of genomic ORFs is the same as for mito ORFs...
Best, Florian
Dear Florian,
I successfully installed Price and all the dependencies and performed the tests with the supplied dataset. While attempting to run Price with my data I got the following error:
2018-04-02 13:34:23.430 INFO Command: gedi -e Price -reads /media/anton/Maxtor/AVS/DATA/A549_RP_mitochondrial/RP_01JUL2015_DEN2_REP1_mitochondrial_032417_sorted.bam -genomic Homo_sapiens_GRCh38_MT -prefix RP_01JUL2015_DEN2_REP1 2018-04-02 13:34:23.566 INFO Discovering classes in classpath 2018-04-02 13:34:23.693 INFO Preparing simple class references 2018-04-02 13:34:23.821 INFO Gedi 1.0.2 (JAR) startup 2018-04-02 13:34:23.928 INFO Reading oml /home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml 2018-04-02 13:34:23.946 INFO Done reading oml /home/anton/.gedi/genomic/Homo_sapiens_GRCh38_MT.oml 2018-04-02 13:34:24.090 INFO Estimating maxpos... 2018-04-02 13:34:24.109 INFO Clustering reads 2018-04-02 13:34:32.416 INFO Found 230 clusters without looking at annotation 2018-04-02 13:34:32.428 INFO Found 9 clusters after looking at annotation 2018-04-02 13:34:32.434 INFO Found 5 clusters after filtering with read/region count PRICE PRICE is an analysis method for Ribo-seq data.
Error: java.lang.RuntimeException: Could not run gedi.riboseq.javapipeline.PriceEstimateModel@72d8e628 Could not run gedi.riboseq.javapipeline.PriceEstimateModel@72d8e628 0
gedi -e PRICE
General: -reads The mapped reads from the ribo-seq experiment.
-prefix The prefix used for all output files
-genomic The indexed GEDI genome.
Commandline: -progress Show progress -D Verbose output of errors -h Show usage -hh Show verbose usage -hhh Show extra verbose usage -dry Dry run -keep Do not remove temp files
any ideas of what might be wrong?
best,
Antón