search

nservant / HiC-Pro

HiC-Pro: An optimized and flexible pipeline for Hi-C data processing
Other
382 stars 183 forks source link

Run HiC-Pro smoothly but can't generate any statistical figures in hic_results/pic/ #382

Closed lugols closed 3 years ago

lugols commented 3 years ago

Hi Nicholas,

I can run HiC-Pro successfully with test data and my own data, the XX_abs.bed and XX_iced.matrix can be used for generating interaction maps. However, there is no files in the hic_results/pic. Do you have any idea for this problem or receive similar issue report? Do you think something is wrong with R-plotting? I found that there is no R-related logs files in the results files.

HiC-Pro version: 2.11.4; R version: 3.4.1; ggplot2 version: 3.1.0;

Here are the testdata log files: build_raw_maps.log ice_500000.log ice_1000000.log mapped_2hic_fragments.log mapping_combine.log mapping_stats.log mapping_step1.log mapping_step2.log merge_stats.log merge_valid_interactions.log mergeSAM.log SRR400264_00_R1_bowtie2.log SRR400264_00_R1_hg19.bwt2glob.unmap_bowtie2.log SRR400264_00_R1_hg19.bwt2glob.unmap_readsTrimming.log SRR400264_00_R2_bowtie2.log SRR400264_00_R2_hg19.bwt2glob.unmap_bowtie2.log SRR400264_00_R2_hg19.bwt2glob.unmap_readsTrimming.log

I am so appreciated for your help!

Best

nservant commented 3 years ago

Hi, Could you please look into the logs folder and the Rout files if there is any error ? Otherwise, I would now suggest to use MultiQC to have the plots, by simply running MultiQC from the HiC-Pro results folder. Best

lugols commented 3 years ago

Hi Nicolas,

Thanks for your reply! I didn't find any Rout files in the logs folder. Do you know or can you guess what may happen?

I have used MultiQC to draw the plots, thanks for the suggestion:)

Best,

Min

From: Nicolas Servant notifications@github.com Sent: Tuesday, December 1, 2020 4:04:15 AM To: nservant/HiC-Pro Cc: Lu, Min/Sloan Kettering Institute; Author Subject: [EXTERNAL] Re: [nservant/HiC-Pro] Run HiC-Pro smoothly but can't generate any statistical figures in hic_results/pic/ (#382)

Hi, Could you please look into the logs folder and the Rout files if there is any error ? Otherwise, I would now suggest to use MultiQC to have the plots, by simply running MultiQC from the HiC-Pro results folder. Best

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/nservant/HiC-Pro/issues/382#issuecomment-736327659, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARZWVYWDNELGPWHKQKZ3SN3SSSWQ7ANCNFSM4T2YPFVQ.

Only open attachments or links from trusted senders. Report phishing to infosec@mskcc.org

=====================================================================

 Please note that this e-mail and any files transmitted from
 Memorial Sloan Kettering Cancer Center may be privileged, confidential,
 and protected from disclosure under applicable law. If the reader of
 this message is not the intended recipient, or an employee or agent
 responsible for delivering this message to the intended recipient,
 you are hereby notified that any reading, dissemination, distribution,
 copying, or other use of this communication or any of its attachments
 is strictly prohibited.  If you have received this communication in
 error, please notify the sender immediately by replying to this message
 and deleting this message, any attachments, and all copies and backups
 from your computer.
nservant commented 3 years ago

humm ... did you run HiC-Pro in stepwise mode with -s ? Otherwise, I do not really see why the plotQC step has been skipped ... without any error ... I guess you already checked that R is up and running ... ?

lugols commented 3 years ago

Hi Nicolas,

I run it in a parallel mode, and I also tried a step-by-step mode, both of them didn't generate any Routput files in Logs.

They can run successfully and generate all results files except for plots.

Wed Oct 21 11:49:31 EDT 2020

Run ICE Normalization ...

Logs: logs/sample1/ice_20000.log

Logs: logs/sample1/ice_40000.log

Logs: logs/sample1/ice_150000.log

Logs: logs/sample1/ice_500000.log

Logs: logs/sample1/ice_1000000.log

Sorry I don't quite understand this very well, I checked the R version (3.4.1) and other necessary packages, and I can run R individually in command line. Do you know how checked R is up and running during HiC-Pro processing?

Best,

Min

From: Nicolas Servant notifications@github.com Sent: Wednesday, December 2, 2020 3:37:56 PM To: nservant/HiC-Pro Cc: Lu, Min/Sloan Kettering Institute; Author Subject: [EXTERNAL] Re: [nservant/HiC-Pro] Run HiC-Pro smoothly but can't generate any statistical figures in hic_results/pic/ (#382)

humm ... did you run HiC-Pro in stepwise mode with -s ? Otherwise, I do not really see why the plotQC step has been skipped ... without any error ... I guess you already checked that R is up and running ... ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/nservant/HiC-Pro/issues/382#issuecomment-737481392, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARZWVYVQWQXFN4XR5U4YPALSS2QSJANCNFSM4T2YPFVQ.

Only open attachments or links from trusted senders. Report phishing to infosec@mskcc.org

=====================================================================

 Please note that this e-mail and any files transmitted from
 Memorial Sloan Kettering Cancer Center may be privileged, confidential,
 and protected from disclosure under applicable law. If the reader of
 this message is not the intended recipient, or an employee or agent
 responsible for delivering this message to the intended recipient,
 you are hereby notified that any reading, dissemination, distribution,
 copying, or other use of this communication or any of its attachments
 is strictly prohibited.  If you have received this communication in
 error, please notify the sender immediately by replying to this message
 and deleting this message, any attachments, and all copies and backups
 from your computer.
nservant commented 3 years ago

Could you try to run the same command with -s quality_checks please ? This step should only run the R scripts ... if there is anything wrong, I guess you should have an error message ...

lugols commented 3 years ago

Hi Nicolas,

Message as below (without ant warning message):

Run HiC-Pro 2.11.4

Wed Dec 2 16:20:51 EST 2020

Run quality checks for all samples ...

No warning/error message. But there is no any files in the logs folder this time.

Best,

Min

From: Nicolas Servant notifications@github.com Sent: Wednesday, December 2, 2020 4:03:04 PM To: nservant/HiC-Pro Cc: Lu, Min/Sloan Kettering Institute; Author Subject: [EXTERNAL] Re: [nservant/HiC-Pro] Run HiC-Pro smoothly but can't generate any statistical figures in hic_results/pic/ (#382)

Could you try to run the same command with -s quality_checks please ? This step should only run the R scripts ... if there is anything wrong, I guess you should have an error message ...

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/nservant/HiC-Pro/issues/382#issuecomment-737493457, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARZWVYROC5P4MXKCTF3AWETSS2TQRANCNFSM4T2YPFVQ.

Only open attachments or links from trusted senders. Report phishing to infosec@mskcc.org

=====================================================================

 Please note that this e-mail and any files transmitted from
 Memorial Sloan Kettering Cancer Center may be privileged, confidential,
 and protected from disclosure under applicable law. If the reader of
 this message is not the intended recipient, or an employee or agent
 responsible for delivering this message to the intended recipient,
 you are hereby notified that any reading, dissemination, distribution,
 copying, or other use of this communication or any of its attachments
 is strictly prohibited.  If you have received this communication in
 error, please notify the sender immediately by replying to this message
 and deleting this message, any attachments, and all copies and backups
 from your computer.
nservant commented 3 years ago

Could you show me the output folder ? do you have a data/ folder with .RSstat files ? do you have a stats folder with .mergestat files for instance ?

lugols commented 3 years ago

Yes, in the parallel/complete mode, there is a sample0_allValidPairs.mergestat and sample0.mRSstat under the path ~/Downloads/hic_results/stats/sample0/.

~/Downloads/201120_results/hic_results/stats/sample0/ contains files:

sample0_allValidPairs.mergestat

sample0_R1.mmapstat

sample0_R2.mmapstat

sample0.mpairstat

sample0.mRSstat

From: Nicolas Servant notifications@github.com Sent: Thursday, December 3, 2020 3:16:35 AM To: nservant/HiC-Pro Cc: Lu, Min/Sloan Kettering Institute; Author Subject: [EXTERNAL] Re: [nservant/HiC-Pro] Run HiC-Pro smoothly but can't generate any statistical figures in hic_results/pic/ (#382)

Could you show me the output folder ? do you have a data/ folder with .RSstat files ? do you have a stats folder with .mergestat files for instance ?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/nservant/HiC-Pro/issues/382#issuecomment-737741173, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARZWVYQNQIJGKXJCB34IW3LSS5COHANCNFSM4T2YPFVQ.

Only open attachments or links from trusted senders. Report phishing to infosec@mskcc.org

=====================================================================

 Please note that this e-mail and any files transmitted from
 Memorial Sloan Kettering Cancer Center may be privileged, confidential,
 and protected from disclosure under applicable law. If the reader of
 this message is not the intended recipient, or an employee or agent
 responsible for delivering this message to the intended recipient,
 you are hereby notified that any reading, dissemination, distribution,
 copying, or other use of this communication or any of its attachments
 is strictly prohibited.  If you have received this communication in
 error, please notify the sender immediately by replying to this message
 and deleting this message, any attachments, and all copies and backups
 from your computer.
nservant commented 3 years ago

I have no idea :( If you want to resolve this mystery, I guess you will need to add traces in the script/make_plots.sh file. Basically, you have a couple of 'if' statement that check if the output folders/files are available. My feeling is that it does not enter into this statement ... but no idea why ...

Otherwise, using the MultiQC files is much nicer ;)

lugols commented 3 years ago

Hi Nicolas,

:) Thank you so much for your time and suggestions!

Best,

Min

From: Nicolas Servant notifications@github.com Sent: Friday, December 4, 2020 3:01:38 AM To: nservant/HiC-Pro Cc: Lu, Min/Sloan Kettering Institute; Author Subject: [EXTERNAL] Re: [nservant/HiC-Pro] Run HiC-Pro smoothly but can't generate any statistical figures in hic_results/pic/ (#382)

I have no idea :( If you want to resolve this mystery, I guess you will need to add traces in the script/make_plots.sh file. Basically, you have a couple of 'if' statement that check if the output folders/files are available. My feeling is that it does not enter into this statement ... but no idea why ...

Otherwise, using the MultiQC files is much nicer ;)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/nservant/HiC-Pro/issues/382#issuecomment-738630359, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARZWVYXPEUP4S6ZUFZ4KHKDSTCJOFANCNFSM4T2YPFVQ.

Only open attachments or links from trusted senders. Report phishing to infosec@mskcc.org

=====================================================================

 Please note that this e-mail and any files transmitted from
 Memorial Sloan Kettering Cancer Center may be privileged, confidential,
 and protected from disclosure under applicable law. If the reader of
 this message is not the intended recipient, or an employee or agent
 responsible for delivering this message to the intended recipient,
 you are hereby notified that any reading, dissemination, distribution,
 copying, or other use of this communication or any of its attachments
 is strictly prohibited.  If you have received this communication in
 error, please notify the sender immediately by replying to this message
 and deleting this message, any attachments, and all copies and backups
 from your computer.