search

PacificBiosciences / pbbioconda

PacBio Secondary Analysis Tools on Bioconda. Contains list of PacBio packages available via conda.
BSD 3-Clause Clear License
251 stars 44 forks source link

pb-assembly error:Task Node(0-rawreads/build) failed with exit-code=1 #424

Closed C2051 closed 3 years ago

C2051 commented 3 years ago

Hi, I come across an issue when I tried to run pb-assembly on linux. My computer is configured with 32G memory， 6 threads. I was using this software to assemble the three-generation sequencing genome of Escherichia coli. But it always reported error as following： [ERROR]Task Node(0-rawreads/build) failed with exit-code=1 [ERROR]Some tasks are recently_done but not satisfied: set([Node(0-rawreads/build)]) [ERROR]ready: set([]) submitted: set([]) [ERROR]Noop. We cannot kill blocked threads. Hopefully, everything will die on SIGTERM. I installed this software using anaconda3. And I create an environment of python 2.7 for its running. And here are the codes below.

(py2) wangdan@DESKTOP-TM5M3VO:~/anaconda3/envs/py2/bin$ fc_run /home/wangdan/anaconda3/envs/py2/ganjun/fc_run_ecoli_local.cfg falcon-kit 1.2.5 pypeflow 2.1.1 [INFO]Setup logging from file "None". [INFO]$ lfs setstripe -c 12 /home/wangdan/anaconda3/envs/py2/bin > [INFO]Apparently '/home/wangdan/anaconda3/envs/py2/bin' is not in lustre filesystem, which is fine. [INFO]fc_run started with configuration /home/wangdan/anaconda3/envs/py2/ganjun/fc_run_ecoli_local.cfg [INFO]cfg= { "General": { "LA4Falcon_preload": false, "avoid_text_file_busy": true, "bestn": 12, "dazcon": false, "falcon_sense_greedy": false, "falcon_sense_option": "--output-multi --min-idt 0.70 --min-cov 2 --max-n-read 800", "falcon_sense_skip_contained": false, "fc_ovlp_to_graph_option": " --min-len 2000", "genome_size": "0", "input_fofn": "input.fofn", "input_type": "raw", "length_cutoff": "3000", "length_cutoff_pr": "2000", "overlap_filtering_setting": "--max-diff 100 --max-cov 300 --min-cov 2", "ovlp_DBdust_option": "", "ovlp_DBsplit_option": "-x500 -s50", "ovlp_HPCdaligner_option": "-v -B128 -M24", "ovlp_daligner_option": "-e.96 -l2000 -k24 -h1024 -w6 -s100", "pa_DBdust_option": "", "pa_DBsplit_option": "-x500 -s50", "pa_HPCTANmask_option": "", "pa_HPCdaligner_option": "-v -B128 -M24", "pa_REPmask_code": "1,100;2,80;3,60", "pa_daligner_option": "-e.7 -l1000 -k18 -h80 -w8 -s100", "pa_dazcon_option": "-j 4 -x -l 500", "pa_fasta_filter_option": "pass", "seed_coverage": "20", "skip_checks": false, "target": "assembly" }, "job.defaults": { "JOB_QUEUE": "default", "MB": "16384", "NPROC": "4", "job_type": "local", "njobs": "4", "pwatcher_type": "blocking", "submit": "/bin/bash -c \"${JOB_SCRIPT}\" > \"${JOB_STDOUT}\" 2> \"${JOB_STDERR}\"", "use_tmpdir": false }, "job.step.asm": { "MB": "16384", "NPROC": "4", "njobs": "4" }, "job.step.cns": { "MB": "16384", "NPROC": "4", "njobs": "5" }, "job.step.da": { "MB": "16384", "NPROC": "4", "njobs": "4" }, "job.step.dust": {}, "job.step.la": { "MB": "16384", "NPROC": "4", "njobs": "4" }, "job.step.pda": {}, "job.step.pla": { "MB": "16384", "NPROC": "4", "njobs": "4" } } [INFO]In simple_pwatcher_bridge, pwatcher_impl=<module 'pwatcher.blocking' from '/home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/pwatcher/blocking.pyc'> [INFO]job_type='local', (default)job_defaults={'JOB_QUEUE': 'default', 'pwatcher_type': 'blocking', 'use_tmpdir': False, 'MB': '16384', 'job_type': 'local', 'submit': '/bin/bash -c "${JOB_SCRIPT}" > "${JOB_STDOUT}" 2> "${JOB_STDERR}"', 'NPROC': '4', 'njobs': '4'}, use_tmpdir=False, squash=False, job_name_style=0 [INFO]Setting max_jobs to 4; was None [INFO]Num unsatisfied: 2, graph: 2 [INFO]About to submit: Node(0-rawreads/build) [INFO]Popen: '/bin/bash -c "/home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/pwatcher/mains/job_start.sh" > "/home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build/run-P198d6046f283bf.bash.stdout" 2> "/home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build/run-P198d6046f283bf.bash.stderr"' [INFO](slept for another 0.0s -- another 1 loop iterations) [INFO](slept for another 0.3s -- another 2 loop iterations) [INFO](slept for another 1.2s -- another 3 loop iterations) [INFO](slept for another 3.0s -- another 4 loop iterations) [INFO](slept for another 6.0s -- another 5 loop iterations) [INFO](slept for another 10.5s -- another 6 loop iterations) [ERROR]Task Node(0-rawreads/build) failed with exit-code=1 [ERROR]Some tasks are recently_done but not satisfied: set([Node(0-rawreads/build)]) [ERROR]ready: set([]) submitted: set([]) [ERROR]Noop. We cannot kill blocked threads. Hopefully, everything will die on SIGTERM. Traceback (most recent call last): File "/home/wangdan/anaconda3/envs/py2/bin/fc_run", line 11, in load_entry_point('falcon-kit==1.2.5', 'console_scripts', 'fc_run')() File "/home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/falcon_kit/mains/run1.py", line 726, in main main1(argv[0], args.config, args.logger) File "/home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/falcon_kit/mains/run1.py", line 76, in main1 input_fofn_fn=input_fofn_fn, File "/home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/falcon_kit/mains/run1.py", line 242, in run dist=Dist(NPROC=4, MB=4000, job_dict=config['job.step.da']), File "/home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/falcon_kit/pype.py", line 106, in gen_parallel_tasks wf.refreshTargets() File "/home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/pypeflow/simple_pwatcher_bridge.py", line 278, in refreshTargets self._refreshTargets(updateFreq, exitOnFailure) File "/home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/pypeflow/simple_pwatcher_bridge.py", line 362, in _refreshTargets raise Exception(msg) Exception: Some tasks are recently_done but not satisfied: set([Node(0-rawreads/build)])

And here is my file run-P198d6046f283bf.bash.stdout. CONDA_DEFAULT_ENV=py2 CONDA_EXE=/home/wangdan/anaconda3/bin/conda CONDA_PREFIX=/home/wangdan/anaconda3/envs/py2 CONDA_PREFIX_1=/home/wangdan/anaconda3 CONDA_PROMPT_MODIFIER=(py2) CONDA_PYTHON_EXE=/home/wangdan/anaconda3/bin/python CONDA_SHLVL=2 HOME=/home/wangdan HOSTTYPE=x86_64 LANG=C.UTF-8 LESSCLOSE=/usr/bin/lesspipe %s %s LESSOPEN=| /usr/bin/lesspipe %s LOGNAME=wangdan LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.zst=01;31:.tzst=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.wim=01;31:.swm=01;31:.dwm=01;31:.esd=01;31:.jpg=01;35:.jpeg=01;35:.mjpg=01;35:.mjpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:*.xspf=00;36: MOTD_SHOWN=update-motd NAME=DESKTOP-TM5M3VO OLDPWD=/home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build PATH=/home/wangdan/anaconda3/envs/py2/bin:/home/wangdan/anaconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Program Files/Common Files/Oracle/Java/javapath_target_358132468:/mnt/c/Windows/System32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/mnt/c/Program Files/MATLAB/R2013b/runtime/win64:/mnt/c/Program Files/MATLAB/R2013b/bin:/mnt/c/Program Files/MEGA-X:/mnt/c/Users/Admin/AppData/Local/Microsoft/WindowsApps:/snap/bin:/~/software/bwa-0.7.12:/~/software/bwa-0.7.12:/bin PWD=/home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build PYPEFLOW_JOB_START_SCRIPT=/home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build/run-P198d6046f283bf.bash PYPEFLOW_JOB_START_TIMEOUT=60 SHELL=/bin/bash SHLVL=4 TERM=xterm-256color USER=wangdan XDG_DATADIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop =/usr/bin/env _CE_CONDA= _CE_M= HOSTNAME=DESKTOP-TM5M3VO PWD=/home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build FAILURE. Running top in /home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build (If you see -terminal database is inaccessible- you are using the python bin-wrapper, so you will not get diagnostic info. No big deal. This process is crashing anyway.) /home/wangdan/anaconda3/envs/py2/bin/python /usr/bin/top top - 15:35:22 up 24 min, 0 users, load average: 0.52, 0.58, 0.59 Tasks: 10 total, 2 running, 8 sleeping, 0 stopped, 0 zombie %Cpu(s): 4.7 us, 4.7 sy, 0.0 ni, 90.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 32700.0 total, 21060.1 free, 11415.9 used, 224.0 buff/cache MiB Swap: 0.0 total, 0.0 free, 0.0 used. 21153.5 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1 root 20 0 10576 616 592 S 0.0 0.0 0:00.10 init 2 wangdan 20 0 29252 3892 3772 S 0.0 0.0 0:00.36 bash 179 wangdan 20 0 233088 37548 36284 S 0.0 0.1 0:00.71 fc_run 183 wangdan 20 0 243744 680 648 S 0.0 0.0 0:00.00 sh 184 wangdan 20 0 277072 1684 1608 S 0.0 0.0 0:00.01 job_start+ 185 wangdan 20 0 293736 1676 1596 S 0.0 0.0 0:00.00 bash 186 wangdan 20 0 310400 1636 1564 S 0.0 0.0 0:00.01 bash 187 wangdan 20 0 327064 1712 1620 S 0.0 0.0 0:00.01 bash 197 wangdan 20 0 361136 2036 1500 R 0.0 0.0 0:00.04 top 198 wangdan 20 0 361136 2000 1464 R 0.0 0.0 0:00.04 top init,1 -bash,2 -fc_run,179 /home/wangdan/anaconda3/envs/py2/bin/fc_run /home/wangdan/anaconda3/envs/py2/ganjun/fc_run_ecoli_local.cfg |-sh,183 -c /bin/bash -c "/home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/pwatcher/mains/job_start.sh" > "/home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build/run-P198d6046f283bf.bash.stdout" 2> "/home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build/run-P198d6046f283bf.bash.stderr" | -job_start.sh,184 /home/wangdan/anaconda3/envs/py2/lib/python2.7/site-packages/pwatcher/mains/job_start.sh |-bash,185 /home/wangdan/anaconda3/envs/py2/bin/0-rawreads/build/run-P198d6046f283bf.bash | -bash,186 run.sh |-bash,187 task.sh | -pstree,199 -apl -{fc_run},182 finish code: 1

I would very appreciate it if you can reply me.

pb-cdunn commented 3 years ago

Please look in the directory 0-rawreads/build/ for stderr/out files. The cause is almost always an integration problem, which is very difficult to debug remotely.