search

cougarlj / COMPSRA

COMPSRA: a COMprehensive Platform for Small RNA-Seq data Analysis
https://regepi.bwh.harvard.edu/circurna/
GNU General Public License v3.0
16 stars 6 forks source link

Annotation error for mm10 based run #11

Closed Hironobu-Murakami closed 3 years ago

Hironobu-Murakami commented 3 years ago

Hello. I am getting the following error for annotation using mm10.

All the best, Murakami

+++++++++++++++++++++++++++++++

DEBUG 2020-12-19 15:18:28 BlockCompressedOutputStream Using deflater: Deflater 15:18:28.029 [pool-2-thread-1] WARN edu.harvard.channing.compass.db.DB_miRBase - The prebuilt database file /Volumes/NGS_1/NGS_tools/COMPSRA_mm/bundle_v1/prebuilt_db/miRBase_mm10.obj doesn't exist! 15:18:28.029 [pool-2-thread-1] INFO edu.harvard.channing.compass.db.DB_miRBase - Try to build the database from local files /Volumes/NGS_1/NGS_tools/COMPSRA_mm/bundle_v1/db/miRBase/miRBase_mm10.gff3 ! 15:18:28.030 [pool-2-thread-1] INFO edu.harvard.channing.compass.core.Factory - Read File Error: Fail to read file: miRBase_v21 15:18:28.030 [pool-2-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - null 15:18:28.031 [pool-2-thread-1] INFO edu.harvard.channing.compass.utility.ReadFile - Read File Error: Fail to read file: /Volumes/NGS_1/NGS_tools/COMPSRA_mm/bundle_v1/db/miRBase/miRBase_mm10.gff3. 15:18:28.031 [pool-2-thread-1] ERROR edu.harvard.channing.compass.core.ann.SAMAnnotator - null java.lang.NullPointerException at edu.harvard.channing.compass.utility.ReadFile.readGFF3(ReadFile.java:384) at edu.harvard.channing.compass.db.DB_miRBase.getForest(DB_miRBase.java:102) at edu.harvard.channing.compass.core.ann.IPTR_miRNA.buildForest(IPTR_miRNA.java:51) at edu.harvard.channing.compass.core.ann.SAMAnnotator.Annotate(SAMAnnotator.java:79) at edu.harvard.channing.compass.core.ann.SAMAnnotator.call(SAMAnnotator.java:160) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 15:18:28.042 [main] ERROR edu.harvard.channing.compass.core.ann.Annotation - java.lang.NullPointerException 15:18:28.042 [main] INFO edu.harvard.channing.compass.core.Produce - Error: Annotation module fails!

+++++++++++++++++++++++++++++++++++++++

cougarlj commented 3 years ago

Dear Murakami,

Please unzip the attached miRBase_mm10.obj.zip file and put it in the directory /Volumes/NGS_1/NGS_tools/COMPSRA_mm/bundle_v1/prebuilt_db/ .

miRBase_mm10.zip

Best Wishes, Jiang Li

Hironobu-Murakami commented 3 years ago

Thank you for your reply. But, the error could not be prevented. In addition, my system could not work annotation using the example file and hg38 database. I have downloaded all prebuild databases (mm10&hg38) and the attached miRbase_mm10.obj.zip file in the specified directory. Could you tell me a solution?

All the best, Murakami

Error mm10

DEBUG 2020-12-22 20:52:53 BlockCompressedOutputStream Using deflater: Deflater 20:52:53.406 [pool-2-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - edu.harvard.channing.compass.entity.DBTree; local class incompatible: stream classdesc serialVersionUID = 556476181312992002, local class serialVersionUID = 1386352989058994328 20:52:53.407 [pool-2-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - Fail to find the prebuilt database obj file! 20:52:53.407 [pool-2-thread-1] WARN edu.harvard.channing.compass.db.DB_miRBase - The prebuilt database file /Volumes/NGS_1/NGS_tools/COMPSRA_V1.0.3/bundle_v1/prebuilt_db/miRBase_hg38.obj doesn't exist! 20:52:53.407 [pool-2-thread-1] INFO edu.harvard.channing.compass.db.DB_miRBase - Try to build the database from local files /Volumes/NGS_1/NGS_tools/COMPSRA_V1.0.3/bundle_v1/db/miRBase/miRBase_hg38.gff3 ! 20:52:53.408 [pool-2-thread-1] INFO edu.harvard.channing.compass.core.Factory - Read File Error: Fail to read file: miRBase_v21 20:52:53.408 [pool-2-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - Cannot invoke "java.io.BufferedReader.readLine()" because "br" is null 20:52:53.408 [pool-2-thread-1] INFO edu.harvard.channing.compass.utility.ReadFile - Read File Error: Fail to read file: /Volumes/NGS_1/NGS_tools/COMPSRA_V1.0.3/bundle_v1/db/miRBase/miRBase_hg38.gff3. 20:52:53.408 [pool-2-thread-1] ERROR edu.harvard.channing.compass.core.ann.SAMAnnotator - Cannot invoke "java.util.HashMap.values()" because "dbt.hmpStart" is null java.lang.NullPointerException: Cannot invoke "java.util.HashMap.values()" because "dbt.hmpStart" is null at edu.harvard.channing.compass.utility.ReadFile.readGFF3(ReadFile.java:384) at edu.harvard.channing.compass.db.DB_miRBase.getForest(DB_miRBase.java:102) at edu.harvard.channing.compass.core.ann.IPTR_miRNA.buildForest(IPTR_miRNA.java:51) at edu.harvard.channing.compass.core.ann.SAMAnnotator.Annotate(SAMAnnotator.java:79) at edu.harvard.channing.compass.core.ann.SAMAnnotator.call(SAMAnnotator.java:160) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) 20:52:53.410 [main] ERROR edu.harvard.channing.compass.core.ann.Annotation - java.lang.NullPointerException: Cannot invoke "edu.harvard.channing.compass.core.ann.IPTR.getReport()" because "this.iptr[i]" is null 20:52:53.410 [main] INFO edu.harvard.channing.compass.core.Produce - Error: Annotation module fails!

Error hg38

DEBUG 2020-12-22 19:22:01 BlockCompressedOutputStream Using deflater: Deflater 19:22:01.508 [pool-2-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - edu.harvard.channing.compass.entity.DBTree; local class incompatible: stream classdesc serialVersionUID = 556476181312992002, local class serialVersionUID = 1386352989058994328 19:22:01.508 [pool-2-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - Fail to find the prebuilt database obj file! 19:22:01.509 [pool-2-thread-1] WARN edu.harvard.channing.compass.db.DB_miRBase - The prebuilt database file /Volumes/NGS_1/NGS_tools/COMPSRA_V1.0.3/bundle_v1/prebuilt_db/miRBase_hg38.obj doesn't exist! 19:22:01.509 [pool-2-thread-1] INFO edu.harvard.channing.compass.db.DB_miRBase - Try to build the database from local files /Volumes/NGS_1/NGS_tools/COMPSRA_V1.0.3/bundle_v1/db/miRBase/miRBase_hg38.gff3 ! 19:22:01.509 [pool-2-thread-1] INFO edu.harvard.channing.compass.core.Factory - Read File Error: Fail to read file: miRBase_v21 19:22:01.509 [pool-2-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - Cannot invoke "java.io.BufferedReader.readLine()" because "br" is null 19:22:01.510 [pool-2-thread-1] INFO edu.harvard.channing.compass.utility.ReadFile - Read File Error: Fail to read file: /Volumes/NGS_1/NGS_tools/COMPSRA_V1.0.3/bundle_v1/db/miRBase/miRBase_hg38.gff3. 19:22:01.510 [pool-2-thread-1] ERROR edu.harvard.channing.compass.core.ann.SAMAnnotator - Cannot invoke "java.util.HashMap.values()" because "dbt.hmpStart" is null java.lang.NullPointerException: Cannot invoke "java.util.HashMap.values()" because "dbt.hmpStart" is null at edu.harvard.channing.compass.utility.ReadFile.readGFF3(ReadFile.java:384) at edu.harvard.channing.compass.db.DB_miRBase.getForest(DB_miRBase.java:102) at edu.harvard.channing.compass.core.ann.IPTR_miRNA.buildForest(IPTR_miRNA.java:51) at edu.harvard.channing.compass.core.ann.SAMAnnotator.Annotate(SAMAnnotator.java:79) at edu.harvard.channing.compass.core.ann.SAMAnnotator.call(SAMAnnotator.java:160) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) 19:22:01.511 [main] ERROR edu.harvard.channing.compass.core.ann.Annotation - java.lang.NullPointerException: Cannot invoke "edu.harvard.channing.compass.core.ann.IPTR.getReport()" because "this.iptr[i]" is null 19:22:01.512 [main] INFO edu.harvard.channing.compass.core.Produce - Error: Annotation module fails!

wei-course commented 3 years ago

I also met the same problem in the annotation step for hg38 DEBUG 2020-12-23 11:06:55 BlockCompressedOutputStream Using deflater: Deflater 11:06:55.171 [pool-4-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - edu.harvard.channing.compass.entity.DBTree; local class incompatible: stream classdesc serialVersionUID = 556476181312992002, local class serialVersionUID = 1386352989058994328 11:06:55.172 [pool-4-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - Fail to find the prebuilt database obj file! 11:06:55.172 [pool-4-thread-1] WARN edu.harvard.channing.compass.db.DB_miRBase - The prebuilt database file /STORAGE/wzhang/RNA-seq/COMPSRA/bundle_v1/prebuilt_db/miRBase_hg38.obj doesn't exist! 11:06:55.172 [pool-4-thread-1] INFO edu.harvard.channing.compass.db.DB_miRBase - Try to build the database from local files /STORAGE/wzhang/RNA-seq/COMPSRA/bundle_v1/db/miRBase/miRBase_hg38.gff3 ! 11:06:55.173 [pool-4-thread-1] INFO edu.harvard.channing.compass.core.Factory - Read File Error: Fail to read file: miRBase_v21 11:06:55.173 [pool-4-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - null 11:06:55.173 [pool-4-thread-1] INFO edu.harvard.channing.compass.utility.ReadFile - Read File Error: Fail to read file: /STORAGE/wzhang/RNA-seq/COMPSRA/bundle_v1/db/miRBase/miRBase_hg38.gff3. 11:06:55.177 [pool-4-thread-1] ERROR edu.harvard.channing.compass.core.ann.SAMAnnotator - null java.lang.NullPointerException at edu.harvard.channing.compass.utility.ReadFile.readGFF3(ReadFile.java:384) at edu.harvard.channing.compass.db.DB_miRBase.getForest(DB_miRBase.java:102) at edu.harvard.channing.compass.core.ann.IPTR_miRNA.buildForest(IPTR_miRNA.java:51) at edu.harvard.channing.compass.core.ann.SAMAnnotator.Annotate(SAMAnnotator.java:79) at edu.harvard.channing.compass.core.ann.SAMAnnotator.call(SAMAnnotator.java:160) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 11:06:55.179 [main] ERROR edu.harvard.channing.compass.core.ann.Annotation - java.lang.NullPointerException 11:06:55.179 [main] INFO edu.harvard.channing.compass.core.Produce - Error: Annotation module fails!

cougarlj commented 3 years ago

Dear Murakami

Please unzip the zip file and get miRBase_mm10.obj to use. All the problems you meet is the version problem. I think, maybe, you can download the latest version of COMPSRA.

I will try to put these pre-built database in GitHub, because our server is not stable for downloading files.

Best Wishes, Jiang Li

Hironobu-Murakami commented 3 years ago

Dear Jiang

Thank you for updating the files. I could succeed to analyze using the hg38 database. However, I have met the error using the mm9&10 databases, again. I hope to be solved the error from the mm9 and mm10.

All the best, Murakami

DEBUG 2020-12-25 11:53:22 BlockCompressedOutputStream Using deflater: Deflater 11:53:23.418 [pool-2-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - edu.harvard.channing.compass.entity.DBTree; local class incompatible: stream classdesc serialVersionUID = 556476181312992002, local class serialVersionUID = 1386352989058994328 11:53:23.419 [pool-2-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - Fail to find the prebuilt database obj file! 11:53:23.419 [pool-2-thread-1] WARN edu.harvard.channing.compass.db.DB_piRNABank - The prebuilt database file /Volumes/NGS_1/NGS_tools/COMPSRA-master/bundle_v1/prebuilt_db/piRNABank_mm10.obj doesn't exist! 11:53:23.419 [pool-2-thread-1] INFO edu.harvard.channing.compass.db.DB_piRNABank - Try to build the database from local files /Volumes/NGS_1/NGS_tools/COMPSRA-master/bundle_v1/db/piRNABank/piRNABank_mm8_all.txt.gz ! 11:53:23.419 [pool-2-thread-1] INFO edu.harvard.channing.compass.db.DB_piRNABank - LiftOver Info: /Volumes/NGS_1/NGS_tools/COMPSRA-master/bundle_v1/db/chain/mm8ToMm10.over.chain.gz is used!

11:53:23.420 [pool-2-thread-1] INFO edu.harvard.channing.compass.utility.ReadFile - Lift Over Error: Fail to read the liftover file: /Volumes/NGS_1/NGS_tools/COMPSRA-master/bundle_v1/db/chain/mm8ToMm10.over.chain.gz 11:53:23.420 [pool-2-thread-1] ERROR edu.harvard.channing.compass.core.ann.SAMAnnotator - Cannot assign field "strDB" because "dbt" is null java.lang.NullPointerException: Cannot assign field "strDB" because "dbt" is null at edu.harvard.channing.compass.db.DB_piRNABank.getForest(DB_piRNABank.java:106) at edu.harvard.channing.compass.core.ann.IPTR_piRNA.buildForest(IPTR_piRNA.java:51) at edu.harvard.channing.compass.core.ann.SAMAnnotator.Annotate(SAMAnnotator.java:79) at edu.harvard.channing.compass.core.ann.SAMAnnotator.call(SAMAnnotator.java:160) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) 11:53:23.425 [main] ERROR edu.harvard.channing.compass.core.ann.Annotation - java.lang.NullPointerException: Cannot invoke "edu.harvard.channing.compass.core.ann.IPTR.getReport()" because "this.iptr[i]" is null 11:53:23.425 [main] INFO edu.harvard.channing.compass.core.Produce - Error: Annotation module fails

cougarlj commented 3 years ago

Dear Murakami,

I have updated the prebuilt databases for mm9 and mm10 and please download them directly from GitHub.

Best Wishes, Jiang

cougarlj commented 3 years ago

Please download the obj files from GitHub.

On Wed, Dec 23, 2020 at 6:56 AM wei-course notifications@github.com wrote:

I also met the same problem in the annotation step for hg38 DEBUG 2020-12-23 11:06:55 BlockCompressedOutputStream Using deflater: Deflater 11:06:55.171 [pool-4-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - edu.harvard.channing.compass.entity.DBTree; local class incompatible: stream classdesc serialVersionUID = 556476181312992002, local class serialVersionUID = 1386352989058994328 11:06:55.172 [pool-4-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - Fail to find the prebuilt database obj file! 11:06:55.172 [pool-4-thread-1] WARN edu.harvard.channing.compass.db.DB_miRBase - The prebuilt database file /STORAGE/wzhang/RNA-seq/COMPSRA/bundle_v1/prebuilt_db/miRBase_hg38.obj doesn't exist! 11:06:55.172 [pool-4-thread-1] INFO edu.harvard.channing.compass.db.DB_miRBase - Try to build the database from local files /STORAGE/wzhang/RNA-seq/COMPSRA/bundle_v1/db/miRBase/miRBase_hg38.gff3 ! 11:06:55.173 [pool-4-thread-1] INFO edu.harvard.channing.compass.core.Factory - Read File Error: Fail to read file: miRBase_v21 11:06:55.173 [pool-4-thread-1] ERROR edu.harvard.channing.compass.utility.ReadFile - null 11:06:55.173 [pool-4-thread-1] INFO edu.harvard.channing.compass.utility.ReadFile - Read File Error: Fail to read file: /STORAGE/wzhang/RNA-seq/COMPSRA/bundle_v1/db/miRBase/miRBase_hg38.gff3. 11:06:55.177 [pool-4-thread-1] ERROR edu.harvard.channing.compass.core.ann.SAMAnnotator - null java.lang.NullPointerException at edu.harvard.channing.compass.utility.ReadFile.readGFF3(ReadFile.java:384) at edu.harvard.channing.compass.db.DB_miRBase.getForest(DB_miRBase.java:102) at edu.harvard.channing.compass.core.ann.IPTR_miRNA.buildForest(IPTR_miRNA.java:51) at edu.harvard.channing.compass.core.ann.SAMAnnotator.Annotate(SAMAnnotator.java:79) at edu.harvard.channing.compass.core.ann.SAMAnnotator.call(SAMAnnotator.java:160) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 11:06:55.179 [main] ERROR edu.harvard.channing.compass.core.ann.Annotation

  • java.lang.NullPointerException 11:06:55.179 [main] INFO edu.harvard.channing.compass.core.Produce - Error: Annotation module fails!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cougarlj/COMPSRA/issues/11#issuecomment-750218988, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGMAD3J3UDP5Y6WDO5DQC3SWHLITANCNFSM4VCCAM7Q .

Hironobu-Murakami commented 3 years ago

Dear Jiang,

Thank you for upload of the files. I could partially analyze my NGS files using mm 10 database. However, I have not been able to completely analyze those because I have got the only miRNA analysis file and have met the below error. I would appreciate it if you could support me.

DEBUG 2021-01-06 13:55:55 BlockCompressedOutputStream Using deflater: Deflater 13:56:34.900 [pool-2-thread-1] INFO edu.harvard.channing.compass.core.ann.IPTR_miRNA - /Volumes/NGS_1/NGS_tools/COMPSRA-master/example_out/sample101/sample101_17to50_FitRead_STAR_Aligned_miRNA.txt was saved. 13:56:35.233 [pool-2-thread-1] ERROR edu.harvard.channing.compass.core.ann.SAMAnnotator - Cannot invoke "java.util.HashMap.keySet()" because "dis.hmpStart" is null java.lang.NullPointerException: Cannot invoke "java.util.HashMap.keySet()" because "dis.hmpStart" is null at edu.harvard.channing.compass.core.ann.IPTR_piRNA.mergeTrees(IPTR_piRNA.java:75) at edu.harvard.channing.compass.core.ann.IPTR_piRNA.writeReport(IPTR_piRNA.java:103) at edu.harvard.channing.compass.core.ann.SAMAnnotator.Annotate(SAMAnnotator.java:112) at edu.harvard.channing.compass.core.ann.SAMAnnotator.call(SAMAnnotator.java:160) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) 13:56:35.562 [main] ERROR edu.harvard.channing.compass.core.ann.Annotation - java.lang.NullPointerException: Cannot invoke "java.util.HashMap.keySet()" because "this.hmpStart" is null 13:56:35.563 [main] INFO edu.harvard.channing.compass.core.Produce - Error: Annotation module fails!

All the best, Murakami

cougarlj commented 3 years ago

Dear Murakami,

Sorry to reply late. According to the message output, it's the problem of piRNA in mouse. We do have the pre-built database for piRNA in mouse, but the size is huge and couldn't upload to GitHub. Also, we still suspect the annotated piRNAs of mouse in piRBase, because there are too many piRNAs. So, the best solution is to ignore piRNA in your analysis, I think.

Best Wishes, Jiang

Hironobu-Murakami commented 3 years ago

Dear Jiang,

Thank you for your supports.

All the best, Murakami