search

Nextomics / NextPolish

Fast and accurately polish the genome generated by long reads.
GNU General Public License v3.0
200 stars 27 forks source link

ImportError #97

Closed mergi-2674 closed 1 year ago

mergi-2674 commented 1 year ago

Describe the bug I was trying to run NextPolish on a cluster on six nodes each with 128GB of RAM. I installed according to the instruction explained on the read the io page. However, it always gives me an error that says,

Error message Traceback (most recent call last): File "/home/users/piwczyn/miniconda3/envs/nextpolish/NextPolish/nextPolish", line 11, in from paralleltask import Task ImportError: No module named "paralleltask",

even if the module is there. I was using a cluster with

Operating system Which operating system and version are you using? redhat-linux-x86_64

GCC What version of GCC are you using? gcc version 4.8.5

Python What version of Python are you using? Python 3.9.12

NextPolish What version of NextPolish are you using? NextPolish version 1.4.1

Thanks!

moold commented 1 year ago

Hello, this is usually caused by using different Pythons between the main and computer nodes. To fix this problem, just use the absolute path of python, try like this /path_to_python/python nextPolish input.cfg

mergi-2674 commented 1 year ago

Hi Hu Jiang, Thank you so much for the fastest reply to my email. It works for me the way you suggested. But it fails with this error. Actually, I wanted to run on 4 nodes. Each node has 48 threads and 128 GB of RAM. Here is my configuration file. Is that right?

[General] job_type = pbs job_prefix = Ferula_nextPolish_flye task = default rewrite = no deltmp = yes rerun = 3 parallel_jobs = 48 genome = /STORAGE/DATA/xxxxx/assembly.fasta genome_size = auto workdir = /STORAGE/DATA/xxxxxx/01_rundir polish_options = -p {multithread_jobs}

[sgs_option] sgs_fofn = /STORAGE/DATA/xxxxxx/sgs.fofn sgs_options = -max_depth 100

[lgs_option] lgs_fofn = /STORAGE/DATA/xxxxxx/lgs.fofn lgs_options = -min_read_len 1k -max_depth 100 lgs_minimap2_options = -x map-ont

and the command line script,

!/bin/bash

PBS -N Nextpolish

PBS -M @.***

PBS -m abe

PBS -l nodes=4:ppn=48

PBS -l mem=128gb

PBS -l walltime=3500:00:00

PBS -o /STORAGE/DATA/xxxx/xxxxx/stdout_${PBS_JOBID}

PBS -e /STORAGE/DATA/xxxx/xxxxx/stderr_${PBS_JOBID}

Cd /STORAGE/DATA/xxxxxxx

/home/users/xxxxxxx/miniconda3/envs/nextpolish/bin/python2.7 /home/users/xxxxxxxx/miniconda3/envs/nextpolish/NextPolish/nextPolish /STORAGE/DATA/xxxxxx/nextpolish.cfg

Best,

Mergi Dinka

E - Mail:- @. @.

On Fri, Jul 29, 2022 at 11:43 AM Hu Jiang @.***> wrote:

Hello, this is usually caused by using different Pythons between the main and computer nodes. To fix this problem, just use the absolute path of python, try like this /path_to_python/python nextPolish input.cfg

— Reply to this email directly, view it on GitHub https://github.com/Nextomics/NextPolish/issues/97#issuecomment-1199080145, or unsubscribe https://github.com/notifications/unsubscribe-auth/APX3PXUXLYFYTHFAUX6HQP3VWORU7ANCNFSM55AD6IOQ . You are receiving this because you authored the thread.Message ID: @.***>

moold commented 1 year ago

It's better to set it like this:

parallel_jobs = 4
multithread_jobs = 48

see here for Parameter introduction.

mergi-2674 commented 1 year ago

Awesome, thanks! Mergi Dinka

E - Mail:- @. @.

On Mon, Aug 1, 2022 at 7:43 AM Hu Jiang @.***> wrote:

Closed #97 https://github.com/Nextomics/NextPolish/issues/97 as completed.

— Reply to this email directly, view it on GitHub https://github.com/Nextomics/NextPolish/issues/97#event-7098223226, or unsubscribe https://github.com/notifications/unsubscribe-auth/APX3PXV3MDHLIIJQF6UQZW3VW5PWLANCNFSM55AD6IOQ . You are receiving this because you authored the thread.Message ID: @.***>