cougarlj / COMPSRA

COMPSRA: a COMprehensive Platform for Small RNA-Seq data Analysis
https://regepi.bwh.harvard.edu/circurna/
GNU General Public License v3.0
16 stars 6 forks source link

STAR installation issue #6

Closed waqasuddinkhan closed 3 years ago

waqasuddinkhan commented 4 years ago

Hi,

I tried to install COMPSRA via COMPSRA command, and also by manually within the plug directory, but no luck: 08:50:48.303 [main] ERROR java.lang.Class - STAR was not installed correctly!

Could you guide me the best possible way,

Thanks in advance,

Waqas.

cougarlj commented 4 years ago

Dear Waqas,

Would you like to provide more output in the screen which can help me to debug it?

Best Wishes, Jiang

waqasuddinkhan commented 4 years ago

Dear Jinag,

I am able to install the STAR, actually the bundle folder should be there where your fastq files are!!!!!!!

Now, I am wondering where the merge count file is saved as I used -fm -fms. The name of my output folder is COMPSRA_OUT. It should be there? or within the sample folder? what would the prefix of the that output file?

Any help is appreciated,

Regards,

Waqas.

waqasuddinkhan commented 4 years ago

Looking forward for your response,

Regards,

Waqas.

cougarlj commented 3 years ago

Dear Waqas,

Sorry to reply so late. I'm engaged in some other projects now. 

In order to manage the plugs, COMPSRA will install all plugs in the directory /bundle_v1/plug/

For a safe input and output, please try to use the full name of each file.

If your output directory is COMPSRA_OUT, I think first COMPSRA will take this as a file prefix and will make a directory in the root directory under your account. So please use the full name of the directory, such as /home/waqas/compsra/compsra_out/ . 

You should get some successful notification on the screen, otherwise the merge function fails. 

Another way to merge COMPSRA small RNA output files is to use the merge function in COMPSRA Toolkit, which already existed in the release, but not mentioned in the manual. The command is like : java -jar COMPSRA.jar -tk -merge -id 0 -count 2 -inf /full/file/name/SampleList.txt -out /full/file/name/miRNA_compsra_merge.txt

-id should set the column of molecular/species name in the annotation file. (Start from 0) -count should set the column of count in the annotation file. (Start from 0) -inf should provide the targeted annotation files (full names). -out provide the output file (full name). 

You may use the find command in Linux to get the annotation file list.  For example, find /annotation/file/directory/ -name *_miRNA.txt | sort > /target/output/directory/COMPSRA_miRNA_sample.txt

Hope these can help you. 

Best Wishes,  Jiang 

PierreSolomon commented 3 years ago

Hello, I also have issues to install star. I tried to move the location of the fastq files to the location of the bundle folder considering Waqas comment but it didn't change anything. Can you help me?

Thanks in advance, Pierre

(base) [psolomon@login-01 COMPSRA]$ java -jar COMPSRA.jar -ref hg38 -aln -mt star -mbi -in ./example_out/sample01/sample01_17to50_FitRead.fastq.gz -out ./example_out/sample01/sample01_17to50_FitRead

+++++++++++++++++++++++++++++++

Working Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA Bundle Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1 Configuration Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/configuration Plug Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/plug N_CPU: 32

Configuration Info: The endogenous database configuration has been set! Configuration Info: 37 databases are set! 10:31:05.584 [main] INFO edu.harvard.channing.compass.core.Configuration - The Configuration was completed!

The hg38 reference genome was set. QC module will not be performed. The hg38 was set in default. Annotation module will not be performed. Microbe module will not be performed. Function module will not be performed.

+++++++++++++++++++++++++++++++

10:31:05.636 [main] ERROR java.lang.Class - STAR was not installed correctly! (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ (base) [psolomon@login-01 COMPSRA]$ java -jar COMPSRA.jar -tk -dr -ck star

+++++++++++++++++++++++++++++++

Working Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA Bundle Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1 Configuration Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/configuration Plug Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/plug N_CPU: 32

Configuration Info: The endogenous database configuration has been set! Configuration Info: 37 databases are set! 10:31:21.098 [main] INFO edu.harvard.channing.compass.core.Configuration - The Configuration was completed!

+++++++++++++++++++++++++++++++

10:31:21.117 [main] INFO edu.harvard.channing.compass.utility.Download - Start to download the resource: https://regepi.bwh.harvard.edu/circurna/bundle_v1/plug/star/STAR-2.5.3a.zip 10:31:31.513 [main] INFO edu.harvard.channing.compass.utility.Download - Fail to download the resource: https://regepi.bwh.harvard.edu/circurna/bundle_v1/plug/star/STAR-2.5.3a.zip java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) at sun.net.www.protocol.https.HttpsClient.(HttpsClient.java:264) at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191) at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:1484) at edu.harvard.channing.compass.utility.Download.download(Download.java:58) at edu.harvard.channing.compass.toolkit.DownloadResource.downloadResource(DownloadResource.java:221) at edu.harvard.channing.compass.toolkit.DownloadResource.runKit(DownloadResource.java:45) at edu.harvard.channing.compass.toolkit.ProduceTK.Dispatch(ProduceTK.java:41) at edu.harvard.channing.compass.core.COMPSRA.main(COMPSRA.java:54) 10:31:31.517 [main] INFO edu.harvard.channing.compass.core.Produce - ToolKit module was completed.

(base) [psolomon@login-01 COMPSRA]$ java -jar COMPSRA.jar -tk -dr -ck star_hg38

+++++++++++++++++++++++++++++++

Working Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA Bundle Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1 Configuration Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/configuration Plug Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/plug N_CPU: 32

Configuration Info: The endogenous database configuration has been set! Configuration Info: 37 databases are set! 10:31:39.275 [main] INFO edu.harvard.channing.compass.core.Configuration - The Configuration was completed!

+++++++++++++++++++++++++++++++

10:31:39.293 [main] INFO edu.harvard.channing.compass.utility.Download - Start to download the resource: http://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz 10:31:49.342 [main] INFO edu.harvard.channing.compass.utility.Download - Fail to download the resource: http://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz java.net.SocketTimeoutException: connect timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at sun.net.NetworkClient.doConnect(NetworkClient.java:175) at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) at sun.net.www.http.HttpClient.(HttpClient.java:242) at sun.net.www.http.HttpClient.New(HttpClient.java:339) at sun.net.www.http.HttpClient.New(HttpClient.java:357) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202) at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:1484) at edu.harvard.channing.compass.utility.Download.download(Download.java:58) at edu.harvard.channing.compass.toolkit.DownloadResource.downloadResource(DownloadResource.java:221) at edu.harvard.channing.compass.toolkit.DownloadResource.runKit(DownloadResource.java:45) at edu.harvard.channing.compass.toolkit.ProduceTK.Dispatch(ProduceTK.java:41) at edu.harvard.channing.compass.core.COMPSRA.main(COMPSRA.java:54) 10:31:49.346 [main] INFO edu.harvard.channing.compass.core.Produce - ToolKit module was completed.

(base) [psolomon@login-01 COMPSRA]$ java -jar COMPSRA.jar -ref hg38 -qc -ra TGGAATTCTCGGGTGCCAAGG -rb 4 -rh 20 -rt 20 -rr 20 -rlh 8,17 -in ./example/sample01.fastq -out ./example_out/

+++++++++++++++++++++++++++++++

Working Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA Bundle Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1 Configuration Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/configuration Plug Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/plug N_CPU: 32

Configuration Info: The endogenous database configuration has been set! Configuration Info: 37 databases are set! 10:32:11.555 [main] INFO edu.harvard.channing.compass.core.Configuration - The Configuration was completed!

The hg38 reference genome was set. Alignment module will not be performed. Annotation module will not be performed. Microbe module will not be performed. Function module will not be performed.

+++++++++++++++++++++++++++++++

10:32:11.608 [main] INFO edu.harvard.channing.compass.core.qc.Fastq - The length of raw read is: 50. 10:32:11.789 [pool-2-thread-1] INFO edu.harvard.channing.compass.core.qc.Fastq -

File:/sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/example/sample01.fastq

10000 reads are processed.

10000 reads are trimed with 3'adapter.

3000 reads are removed because of low quality (<20).

0 reads are trimed from 5'end with 0 bases.

0 reads are trimed from 3'end with 0 bases.

51 reads are trimed from 5'end bacause of low quality.

38 reads are trimed from 3'end bacause of low quality.

3000 reads are saved int the file: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/example_out/sample01/sample01_LowQualityRead.fastq.gz

247 reads are saved int the file: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/example_out/sample01/sample01_0to8_FitRead.fastq.gz

3303 reads are saved int the file: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/example_out/sample01/sample01_8to17_FitRead.fastq.gz

3450 reads are saved int the file: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/example_out/sample01/sample01_17to50_FitRead.fastq.gz

The length distribution of the trimed reads is listed:

Length Count Percentage

0 3000 30.00% 1 74 0.74% 2 9 0.09% 3 0 0.00% 4 10 0.10% 5 17 0.17% 6 51 0.51% 7 86 0.86% 8 143 1.43% 9 197 1.97% 10 429 4.29% 11 652 6.52% 12 450 4.50% 13 614 6.14% 14 325 3.25% 15 285 2.85% 16 208 2.08% 17 182 1.82% 18 152 1.52% 19 171 1.71% 20 170 1.70% 21 411 4.11% 22 477 4.77% 23 262 2.62% 24 122 1.22% 25 88 0.88% 26 79 0.79% 27 92 0.92% 28 83 0.83% 29 88 0.88% 30 93 0.93% 31 144 1.44% 32 175 1.75% 33 62 0.62% 34 39 0.39% 35 27 0.27% 36 15 0.15% 37 39 0.39% 38 36 0.36% 39 82 0.82% 40 148 1.48% 41 213 2.13% 42 0 0.00% 43 0 0.00% 44 0 0.00% 45 0 0.00% 46 0 0.00% 47 0 0.00% 48 0 0.00% 49 0 0.00% 50 0 0.00%

File /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/example/sample01.fastq QC pass! 10:32:11.997 [main] INFO edu.harvard.channing.compass.core.Produce - QC module was completed.

+++++++++++++++++++++++++++++++++++++++

Fri Dec 11 10:32:11 CET 2020 (base) [psolomon@login-01 COMPSRA]$ java -jar COMPSRA.jar -ref hg38 -aln -mt star -mbi -in ./example_out/sample01/sample01_17to50_FitRead.fastq.gz -out ./example_out/sample01/sample01_17to50_FitRead

+++++++++++++++++++++++++++++++

Working Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA Bundle Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1 Configuration Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/configuration Plug Directory: /sandbox/shares/epigenetics/PSYCOURSE/COMPSRA/bundle_v1/plug N_CPU: 32

Configuration Info: The endogenous database configuration has been set! Configuration Info: 37 databases are set! 10:32:21.192 [main] INFO edu.harvard.channing.compass.core.Configuration - The Configuration was completed!

The hg38 reference genome was set. QC module will not be performed. The hg38 was set in default. Annotation module will not be performed. Microbe module will not be performed. Function module will not be performed.

+++++++++++++++++++++++++++++++

10:32:21.246 [main] ERROR java.lang.Class - STAR was not installed correctly! (base) [psolomon@login-01 COMPSRA]$

cougarlj commented 3 years ago

STAR-2.5.3a.zip Dear PierreSolomon,

Sorry for that. Our server is very slow and not fit for file transfer. I have attached the STAR zip file for you and you can also download it from the official website.

Best Wishes, Jiang Li