moold
commented
2 years ago check the rewrite option, if you set rewrite == no, nextPolish will bakcup the working directory to NEXTPOLISH.backup%d if it already existed, you also can change the working directory by the workdir option.
Describe the bug Hi, I am getting an error during when attempting to run NextPolish, and when I check the log file it indicates, it tells me a variety of files cannot be found. I have the genome and config file in the same directory (NEXTPOLISH) when I start running the program, but once it starts running, it seems to rename this directory to NEXTPOLISH.backup0 and use the old directory name to run the scripts etc. However, based on the log files below, it seems to be looking for the genome and config file in the NEXTPOLISH directory still, even though it has moved them to NEXTPOLISH.backup0. I am not sure why it is doing this or how to fix this, and I am hoping you can advise.
Error message Main task log: [39497 INFO] 2022-04-26 11:05:25 NextPolish start... [39497 INFO] 2022-04-26 11:05:25 version:v1.4.0 logfile:pid39497.log.info [39497 WARNING] 2022-04-26 11:05:25 Delete task: 5 due to missing lgs_fofn. [39497 WARNING] 2022-04-26 11:05:25 Delete task: 5 due to missing lgs_fofn. [39497 WARNING] 2022-04-26 11:05:25 Delete task: 6 due to missing hifi_fofn. [39497 WARNING] 2022-04-26 11:05:25 Delete task: 6 due to missing hifi_fofn. [39497 INFO] 2022-04-26 11:05:25 scheduled tasks: [1, 2, 1, 2] [39497 INFO] 2022-04-26 11:05:25 options: [39497 INFO] 2022-04-26 11:05:25 rerun: 3 rewrite: 0 kill: None cleantmp: 0 use_drmaa: 0 submit: None job_type: local sgs_unpaired: 0 sgs_rm_nread: 1 lgs_read_type: parallel_jobs: 6 align_threads: 5 check_alive: None task: [1, 2, 1, 2] job_id_regex: None sgs_max_depth: 100 lgs_max_depth: 100 multithread_jobs: 5 lgs_max_read_len: 0 hifi_max_depth: 100 lgs_block_size: 500M lgs_min_read_len: 1k hifi_max_read_len: 0 genome_size: 30468886 polish_options: -p 5 hifi_block_size: 500M hifi_min_read_len: 1k job_prefix: nextPolish sgs_block_size: 500000000 sgs_use_duplicate_reads: 0 lgs_minimap2_options: -x map-ont hifi_minimap2_options: -x map-pb sgs_align_options: bwa mem -p -t 5 workdir: /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH sgs_fofn: /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/sgs.fofn snp_phase: /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/%02d.snp_phase snp_valid: /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/%02d.snp_valid lgs_polish: /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/%02d.lgs_polish kmer_count: /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/%02d.kmer_count hifi_polish: /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/%02d.hifi_polish score_chain: /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/%02d.score_chain genome: /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/meteora_purged.medaka.2.fa [39497 WARNING] 2022-04-26 11:05:25 mv /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH to /misc/scratch4/kelsey/meteora_genome/MinION/POLIS HING/PURGED_ASSEMBLY/NEXTPOLISH.backup0 [39497 INFO] 2022-04-26 11:05:25 step 0 and task 1 start: [39497 INFO] 2022-04-26 11:05:30 Total jobs: 3 [39497 INFO] 2022-04-26 11:05:30 Submitted jobID:[39514] jobCmd:[/misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh] in the local_cycle. [39514 CRITICAL] 2022-04-26 11:05:30 Command '/bin/sh /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh > /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh.o 2> /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh.e' returned non-zero exit status 1, error info: . [39497 INFO] 2022-04-26 11:05:31 Submitted jobID:[39519] jobCmd:[/misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh] in the local_cycle. [39519 CRITICAL] 2022-04-26 11:05:31 Command '/bin/sh /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh > /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh.o 2> /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh.e' returned non-zero exit status 1, error info: . [39497 INFO] 2022-04-26 11:05:31 Submitted jobID:[39524] jobCmd:[/misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh] in the local_cycle. [39524 CRITICAL] 2022-04-26 11:05:31 Command '/bin/sh /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh > /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh.o 2> /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh.e' returned non-zero exit status 1, error info: . [39497 ERROR] 2022-04-26 11:05:38 db_split failed: please check the following logs: [39497 ERROR] 2022-04-26 11:05:38 /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh.e [39497 ERROR] 2022-04-26 11:05:38 /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh.e [39497 ERROR] 2022-04-26 11:05:38 /misc/scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh.e
Failed subtask logs: hostname
real 0m0.003s user 0m0.002s sys 0m0.000s
hostname
real 0m0.002s user 0m0.000s sys 0m0.001s
hostname
real 0m0.004s user 0m0.000s sys 0m0.003s
Operating system Distributor ID: Ubuntu Description: Ubuntu 18.04 LTS Release: 18.04 Codename: bionic
GCC gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)
Python Python 3.10.4
NextPolish nextPolish v1.4.0
Additional info I am using a conda environment to run this, if that is helpful/relevant Directory listings after running: (nextpolish) /scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH.backup0@perun16> ll total 21M -rw-r--r-- 1 kelsey roger 30M Apr 26 10:08 meteora_purged.medaka.2.fa -rw-r--r-- 1 kelsey roger 541 Apr 26 10:12 nextpolish.sh -rw-r--r-- 1 kelsey roger 194 Apr 26 10:12 sgs.fofn -rw-r--r-- 1 kelsey roger 68 Apr 26 10:12 run.cfg -rw-r--r-- 1 kelsey roger 6.0K Apr 26 11:05 pid39497.log.info (nextpolish) /scratch4/kelsey/meteora_genome/MinION/POLISHING/PURGED_ASSEMBLY/NEXTPOLISH@perun16> ll total 24K drwxr-xr-x 3 kelsey roger 5 Apr 26 11:05 00.score_chain