stuck on process to cat files

[ryao-mdanderson]

Hi Juicer support team:

I have to modify juicer.sh to adapt it to run on our LSF cluster. The test run completed.

When I run against my dataset (273G for each fastq), the job is pending on the stage Job <664648>, Job Name , User , Project , Service Class , Status , Queue , Command < cat /rsrch3/scratch/itops/ryao/juicer/WM115/aligned//a1607 556732_msplit_optdups.txt > /rsrch3/scratch/itops/ryao/ju icer/WM115/aligned/opt_dups.txt; cat /rsrch3/scratch/itop s/ryao/juicer/WM115/aligned//a1607556732_msplit_dups.txt

/rsrch3/scratch/itops/ryao/juicer/WM115/aligned/dups.txt ;cat /rsrch3/scratch/itops/ryao/juicer/WM115/aligned//a160 7556732_msplit*_merged_nodups.txt > /rsrch3/scratch/itops/ ryao/juicer/WM115/aligned/merged_nodups.txt>, Esub <minrun limit jobinfo>

in aligned directory, it has 1042 files for each types of files: msplit_optdups.txt msplit_dups.txt msplit_merged_nodups.txt but they can't be concatenated to an output file. Do you have any suggestion?

Thanks! Rong Yao

[nchernia]

I would try to just run that command (i.e. kill the jobs and run that command on the command line). If it works, you can restart the pipeline with the "-S final" flag.

On Thu, Dec 10, 2020 at 10:46 AM ryao-mdanderson notifications@github.com wrote:

Hi Juicer support team:

I have to modify juicer.sh to adapt it to run on our LSF cluster. The test run completed.

When I run against my dataset (273G for each fastq), the job is pending on the stage Job <664648>, Job Name , User , Project , Service Class , Status , Queue , Command < cat /rsrch3/scratch/itops/ryao/juicer/WM115/aligned//a1607 556732_msplit_optdups.txt > /rsrch3/scratch/itops/ryao/ju icer/WM115/aligned/opt_dups.txt; cat /rsrch3/scratch/itop s/ryao/juicer/WM115/aligned//a1607556732_msplit_dups.txt

/rsrch3/scratch/itops/ryao/juicer/WM115/aligned/dups.txt ;cat /rsrch3/scratch/itops/ryao/juicer/WM115/aligned//a160 7556732_msplit*_merged_nodups.txt > /rsrch3/scratch/itops/ ryao/juicer/WM115/aligned/merged_nodups.txt>, Esub

in aligned directory, it has 1042 files for each types of files: msplit_optdups.txt msplit_dups.txt msplit_merged_nodups.txt but they can't be concatenated to an output file. Do you have any suggestion?

Thanks! Rong Yao

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/aidenlab/juicer/issues/195, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK2EWY5FQXVYS7DZQXR7LTSUDUOPANCNFSM4UVGOI3Q .

-- Neva Cherniavsky Durand, Ph.D. | she, her, hers Assistant Professor | Molecular and Human Genetics Aiden Lab | Baylor College of Medicine www.aidenlab.org

[ryao-mdanderson]

Thanks for the suggestion. I ran the 3 cat command lines, but out of disk space at the end. May I remove directory splits (1.7 Terabytes) at this point before running cat? What is the possible reasons that pipeline stuck at cat step?

Thanks! Rong

[nchernia]

It is stuck because it is running out of disk space.

You can delete files ending with fastq at this point from the splits folder; see if that helps. If not you can delete sam but I hesitate just because you haven’t completed the dedupping step yet.

On Fri, Dec 11, 2020 at 4:22 PM ryao-mdanderson notifications@github.com wrote:

Thanks for the suggestion. I ran the 3 cat command lines, but out of disk space at the end. May I remove directory splits (1.7 Terabytes) at this point before running cat? What is the possible reasons that pipeline stuck at cat step?

Thanks!

Rong

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/aidenlab/juicer/issues/195#issuecomment-743433375, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK2EW5C22ZWWNNPOJSAERLSUKEQ3ANCNFSM4UVGOI3Q .

-- Neva Cherniavsky Durand, Ph.D. | she, her, hers Assistant Professor | Molecular and Human Genetics Aiden Lab | Baylor College of Medicine www.aidenlab.org