nchernia
commented
3 years ago It probably never got it in the first place.
I would kill that job (make sure you keep track of which split it’s running on).
Then use the below to remove dups just on that file. Then concatenate all the merged_dups, dups, and opt_dups (via cat) into the three big files. And relaunch juicer at -S final.
# Usage:awk -v nodupfile="nodups" -v dupfile="dups" -f rmdups.awk
Reads infile, writes two files, "nodups" and "dups", where the duplicates
are stored in dups.
absolute value
function abs(v) {
return v<0?-v:v;}
for every line
BEGIN {
dupname=name"dups.txt";
nodupname=name"merged_nodups.txt";}
{
if strand, chromosome match previous line and both positions are within 10
it's a dup, otherwise nodup
if ($1!=p1 || $2 != p2 || $3 != p3 || $4 != p4 || $5 != p5 || $6!= p6 || $7 != p7 || $8 != p8){
print > nodupname
}
else {
print > dupname
}}
assign previous whether dup or nodup
{
p1=$1;p2=$2;p3=$3;p4=$4;p5=$5;p6=$6;p7=$7;p8=$8}
On Fri, Mar 19, 2021 at 1:58 PM Dustin Sokolowski @.***> wrote:
Thank you for this great tool.
I am running the juicer.sh pipeline using the PBS version and I am on the deduplication step. It submits about 100 jobs, 99 of them finish in ~10 hours. There is one final job however that has been running for 130+ hours and shows no signs of stopping. I looked into an type previous error that may have this issue and the closest I saw was to add a "j" flag (#169 https://github.com/aidenlab/juicer/issues/169). When I tried to reproduce this, I see that -j is now an illegal flag. Is it possible that this flag was not carried over to the PBS version (or is no longer needed?). Otherwise, is there a way to speed up the deduplication step or cancel this single job and let the remaining 99+% of the Hi-C data deduplicate and align?
Thanks so much, Dustin
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/aidenlab/juicer/issues/214, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK2EW3TZID7ORYRVI7DUD3TEOGE3ANCNFSM4ZPH5TEQ .
-- Neva Cherniavsky Durand, Ph.D. | she, her, hers Assistant Professor | Molecular and Human Genetics Aiden Lab | Baylor College of Medicine www.aidenlab.org
DustinSokolowski
Thank you for this great tool.
I am running the juicer.sh pipeline using the PBS version and I am on the deduplication step. It submits about 100 jobs, 99 of them finish in ~10 hours. There is one final job however that has been running for 130+ hours and shows no signs of stopping. I looked into an type previous error that may have this issue and the closest I saw was to add a "j" flag (https://github.com/aidenlab/juicer/issues/169). When I tried to reproduce this, I see that -j is now an illegal flag. Is it possible that this flag was not carried over to the PBS version (or is no longer needed?). Otherwise, is there a way to speed up the deduplication step or cancel this single job and let the remaining 99+% of the Hi-C data deduplicate and align?
Thanks so much, Dustin