search

PacificBiosciences / FALCON

FALCON: experimental PacBio diploid assembler -- Out-of-date -- Please use a binary release: https://github.com/PacificBiosciences/FALCON_unzip/wiki/Binaries
https://github.com/PacificBiosciences/FALCON_unzip/wiki/Binaries
Other
205 stars 102 forks source link

How to restart after job in 0-rawreads/m_00* failed and the reason? #513

Open lamz138138 opened 7 years ago

lamz138138 commented 7 years ago

Hi!

My falcon was ran as "fc_run.py fc_run.cfg >fc_run.log 2>fc_run.err &". After it exit, I tried to find the reason by "grep 'failed' fc_run.err", then I found the two failed job: 0-rawreads/m_00047 and 0-rawreads/m_00090. After checking 0-rawreads/m_00047/pwatcher.dir/stderr, I found "LAmerge: Cannot open ./L1.47.100.las for 'r'", then I "rm 0-rawreads/m_00047/pwatcher.dir && rm 0-rawreads/m_00090/pwatcher.dir" and restart fc_run.py, but it exit with the same error. I further "rm 0-rawreads/m && rm mypwatcher" and restart fcrun.py, now all new "0-rawreads/m job were failed for lacking of *las files.

1) What is the right way to restart my job? I think I don't need to delete all m* jobs. 2) In fact, las files such as L1.47.100.las could be found in 0-rawreads/job*, why it complain couldn't found such files? 3) Suppose "use_tmpdir = /Project/Genome_Assembly/PacBio/Stage_1/tmp", then I found a lot of las files in /Project/Genome_Assembly/PacBio/Stage_1/tmp/0-rawreads/m_00047 except " L1.47.100.las", the 0-rawreads/m_00047/pwatcher.dir/stderr was as follow. By the way, the path of "../../../../" is right.

Any suggestion would be grateful!

+ python2.7 -m pwatcher.mains.fs_heartbeat --directory=/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047 --heartbeat-file=/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/mypwatcher/heartbeats/heartbeat-P9b2f6d0a252ff9 --exit-file=/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/mypwatcher/exits/exit-P9b2f6d0a252ff9 --rate=10.0 /bin/bash run.sh
Namespace(command=['/bin/bash', 'run.sh'], directory='/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047', exit_file='/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/mypwatcher/exits/exit-P9b2f6d0a252ff9', heartbeat_file='/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/mypwatcher/heartbeats/heartbeat-P9b2f6d0a252ff9', rate=10.0)

cwd:'/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047'
hostname=pluto
heartbeat_fn='/Project/Genome_Assembly/PacBio/Stage_1/mypwatcher/heartbeats/heartbeat-P9b2f6d0a252ff9'
exit_fn='/Project/Genome_Assembly/PacBio/Stage_1/mypwatcher/exits/exit-P9b2f6d0a252ff9'
sleep_s=10.0
before setpgid: pid=13088 pgid=1039
 after setpgid: pid=13088 pgid=13088
In cwd: /Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047, Blocking call: '/bin/bash run.sh'
cd /Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047
+ cd /Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047
bash task.sh
+ bash task.sh
2017-01-19 17:12:30,062 - root - DEBUG - Running "/Project/Genome_Assembly/PacBio/Tools/Falcon/FALCON-integrate/pypeFLOW/pypeflow/do_task.py --tmpdir /rd1/user/Project/Genome_Assembly/PacBio/Stage_1/tmp /rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/task.json"
2017-01-19 17:12:30,063 - root - DEBUG - Checking existence of '/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/task.json' with timeout=60
2017-01-19 17:12:30,063 - root - DEBUG - Loading JSON from '/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/task.json'
2017-01-19 17:12:30,065 - root - DEBUG - {u'inputs': {u'gathered_las': u'/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/raw-gather/gathered_las.txt',
             u'scatter_fn': u'/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/merge-scatter/scattered.json'},
 u'outputs': {u'job_done': u'm_00047_done',
              u'merged_las': u'raw_reads.47.las'},
 u'parameters': {u'config': {u'avoid_text_file_busy': True,
                             u'cns_concurrent_jobs': 25,
                             u'dazcon': False,
                             u'dust': False,
                             u'falcon_sense_greedy': False,
                             u'falcon_sense_option': u'--output_multi --min_idt 0.70 --min_cov 3 --max_n_read 200 --n_core 4',
                             u'falcon_sense_skip_contained': False,
                             u'fc_ovlp_to_graph_option': u' --min_len 8000',
                             u'genome_size': 3000000000,
                             u'input_fofn': u'input.fofn',
                             u'input_type': u'raw',
                             u'install_prefix': u'/Project/Genome_Assembly/PacBio/Tools/Falcon/myVirtualenv',
                             u'job_queue': u'smpQ',
                             u'job_type': u'local',
                             u'length_cutoff': 6000,
                             u'length_cutoff_pr': 8000,
                             u'overlap_filtering_setting': u'--max_diff 40 --max_cov 80 --min_cov 2 --min_len 1 --n_core 100',
                             u'ovlp_DBsplit_option': u'-s400',
                             u'ovlp_HPCdaligner_option': u'-v -B4 -t32 -k24 -e.96 -s100 -M25 -l2500 -h1024',
                             u'ovlp_concurrent_jobs': 20,
                             u'pa_DBdust_option': u'-w128 -t2.5 -m20',
                             u'pa_DBsplit_option': u'-x500 -s400',
                             u'pa_HPCdaligner_option': u'-v -B4 -t16 -k18 -e.70 -s100 -M25 -l4800 -h480 -w8',
                             u'pa_concurrent_jobs': 20,
                             u'pa_dazcon_option': u'-j 4 -x -l 500',
                             u'pwatcher_directory': u'mypwatcher',
                             u'pwatcher_type': u'fs_based',
                             u'seed_coverage': 3.0,
                             u'sge_option': u'-V -pe smp 8 -q smpQ',
                             u'sge_option_cns': u'-V -pe smp 8 -q smpQ',
                             u'sge_option_da': u'-V -pe smp 8 -q smpQ',
                             u'sge_option_fc': u'-V -pe smp 24 -q smpQ',
                             u'sge_option_la': u'-V -pe smp 2 -q smpQ',
                             u'sge_option_pda': u'-V -pe smp 8 -q smpQ',
                             u'sge_option_pla': u'-V -pe smp 2 -q smpQ',
                             u'skip_checks': True,
                             u'stop_all_jobs_on_failure': False,
                             u'target': u'assembly',
                             u'use_tmpdir': u'/Project/Genome_Assembly/PacBio/Stage_1/tmp'},
                 u'job_id': 47,
                 u'merge_script': u'\n# Get the real path of a link, following all links (since \'readlink -f\' is not on OSX).\n# http://stackoverflow.com/questions/7665/how-to-resolve-symbolic-links-in-a-shell-script/697552\nrealpath() {\n    local r=$1; local t=$(readlink $r)\n    while [ $t ]; do\n        r=$(cd $(dirname $r) && cd $(dirname $t) && pwd -P)/$(basename $t)\n        t=$(readlink $r)\n    done\n    echo $r\n}\nrmfollow() {\n    local path=$(realpath $1)\n    rm -f "${path}"\n}\nLAmerge -v raw_reads.47 L1.47.1 L1.47.2 L1.47.3 L1.47.4 L1.47.5 L1.47.6 L1.47.7 L1.47.8 L1.47.9 L1.47.10 L1.47.11 L1.47.12 L1.47.13 L1.47.14 L1.47.15 L1.47.16 L1.47.17 L1.47.18 L1.47.19 L1.47.20 L1.47.21 L1.47.22 L1.47.23 L1.47.24 L1.47.25 L1.47.26 L1.47.27 L1.47.28 L1.47.29 L1.47.30 L1.47.31 L1.47.32 L1.47.33 L1.47.34 L1.47.35 L1.47.36 L1.47.37 L1.47.38 L1.47.39 L1.47.40 L1.47.41 L1.47.42 L1.47.43 L1.47.44 L1.47.45 L1.47.46 L1.47.47 L1.47.48 L1.47.49 L1.47.50 L1.47.51 L1.47.52 L1.47.53 L1.47.54 L1.47.55 L1.47.56 L1.47.57 L1.47.58 L1.47.59 L1.47.60 L1.47.61 L1.47.62 L1.47.63 L1.47.64 L1.47.65 L1.47.66 L1.47.67 L1.47.68 L1.47.69 L1.47.70 L1.47.71 L1.47.72 L1.47.73 L1.47.74 L1.47.75 L1.47.76 L1.47.77 L1.47.78 L1.47.79 L1.47.80 L1.47.81 L1.47.82 L1.47.83 L1.47.84 L1.47.85 L1.47.86 L1.47.87 L1.47.88 L1.47.89 L1.47.90 L1.47.91 L1.47.92 L1.47.93 L1.47.94 L1.47.95 L1.47.96 L1.47.97 L1.47.98 L1.47.99 L1.47.100\nrmfollow L1.47.1.las\nrmfollow L1.47.2.las\nrmfollow L1.47.3.las\nrmfollow L1.47.4.las\nrmfollow L1.47.5.las\nrmfollow L1.47.6.las\nrmfollow L1.47.7.las\nrmfollow L1.47.8.las\nrmfollow L1.47.9.las\nrmfollow L1.47.10.las\nrmfollow L1.47.11.las\nrmfollow L1.47.12.las\nrmfollow L1.47.13.las\nrmfollow L1.47.14.las\nrmfollow L1.47.15.las\nrmfollow L1.47.16.las\nrmfollow L1.47.17.las\nrmfollow L1.47.18.las\nrmfollow L1.47.19.las\nrmfollow L1.47.20.las\nrmfollow L1.47.21.las\nrmfollow L1.47.22.las\nrmfollow L1.47.23.las\nrmfollow L1.47.24.las\nrmfollow L1.47.25.las\nrmfollow L1.47.26.las\nrmfollow L1.47.27.las\nrmfollow L1.47.28.las\nrmfollow L1.47.29.las\nrmfollow L1.47.30.las\nrmfollow L1.47.31.las\nrmfollow L1.47.32.las\nrmfollow L1.47.33.las\nrmfollow L1.47.34.las\nrmfollow L1.47.35.las\nrmfollow L1.47.36.las\nrmfollow L1.47.37.las\nrmfollow L1.47.38.las\nrmfollow L1.47.39.las\nrmfollow L1.47.40.las\nrmfollow L1.47.41.las\nrmfollow L1.47.42.las\nrmfollow L1.47.43.las\nrmfollow L1.47.44.las\nrmfollow L1.47.45.las\nrmfollow L1.47.46.las\nrmfollow L1.47.47.las\nrmfollow L1.47.48.las\nrmfollow L1.47.49.las\nrmfollow L1.47.50.las\nrmfollow L1.47.51.las\nrmfollow L1.47.52.las\nrmfollow L1.47.53.las\nrmfollow L1.47.54.las\nrmfollow L1.47.55.las\nrmfollow L1.47.56.las\nrmfollow L1.47.57.las\nrmfollow L1.47.58.las\nrmfollow L1.47.59.las\nrmfollow L1.47.60.las\nrmfollow L1.47.61.las\nrmfollow L1.47.62.las\nrmfollow L1.47.63.las\nrmfollow L1.47.64.las\nrmfollow L1.47.65.las\nrmfollow L1.47.66.las\nrmfollow L1.47.67.las\nrmfollow L1.47.68.las\nrmfollow L1.47.69.las\nrmfollow L1.47.70.las\nrmfollow L1.47.71.las\nrmfollow L1.47.72.las\nrmfollow L1.47.73.las\nrmfollow L1.47.74.las\nrmfollow L1.47.75.las\nrmfollow L1.47.76.las\nrmfollow L1.47.77.las\nrmfollow L1.47.78.las\nrmfollow L1.47.79.las\nrmfollow L1.47.80.las\nrmfollow L1.47.81.las\nrmfollow L1.47.82.las\nrmfollow L1.47.83.las\nrmfollow L1.47.84.las\nrmfollow L1.47.85.las\nrmfollow L1.47.86.las\nrmfollow L1.47.87.las\nrmfollow L1.47.88.las\nrmfollow L1.47.89.las\nrmfollow L1.47.90.las\nrmfollow L1.47.91.las\nrmfollow L1.47.92.las\nrmfollow L1.47.93.las\nrmfollow L1.47.94.las\nrmfollow L1.47.95.las\nrmfollow L1.47.96.las\nrmfollow L1.47.97.las\nrmfollow L1.47.98.las\nrmfollow L1.47.99.las\nrmfollow L1.47.100.las\n',
                 u'sge_option': u'-V -pe smp 2 -q smpQ'},
 u'python_function': u'falcon_kit.pype_tasks.task_run_las_merge'}
2017-01-19 17:12:30,065 - root - DEBUG - CD: '/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047' <- '/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047'
2017-01-19 17:12:30,065 - root - DEBUG - Checking existence of u'/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/raw-gather/gathered_las.txt' with timeout=60
2017-01-19 17:12:30,065 - root - DEBUG - Checking existence of u'/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/merge-scatter/scattered.json' with timeout=60
2017-01-19 17:12:30,078 - root - INFO - mkdir -p /Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp//rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047
2017-01-19 17:12:30,081 - root - DEBUG - CD: '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp//rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047' <- '/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047'
2017-01-19 17:12:30,093 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_050f/L1.47.86.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.86.las'
2017-01-19 17:12:30,094 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_047f/L1.47.56.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.56.las'
2017-01-19 17:12:30,094 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0106/L1.47.80.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.80.las'
2017-01-19 17:12:30,094 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0507/L1.47.62.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.62.las'
2017-01-19 17:12:30,094 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03d0/L1.47.87.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.87.las'
2017-01-19 17:12:30,094 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03d6/L1.47.10.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.10.las'
2017-01-19 17:12:30,094 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03d6/L1.47.11.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.11.las'
2017-01-19 17:12:30,095 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03d6/L1.47.8.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.8.las'
2017-01-19 17:12:30,095 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03d6/L1.47.9.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.9.las'
2017-01-19 17:12:30,095 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03d5/L1.47.54.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.54.las'
2017-01-19 17:12:30,095 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01b5/L1.47.28.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.28.las'
2017-01-19 17:12:30,095 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01b5/L1.47.29.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.29.las'
2017-01-19 17:12:30,095 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01b5/L1.47.30.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.30.las'
2017-01-19 17:12:30,096 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01b5/L1.47.31.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.31.las'
2017-01-19 17:12:30,096 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_04ac/L1.47.69.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.69.las'
2017-01-19 17:12:30,096 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_027a/L1.47.85.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.85.las'
2017-01-19 17:12:30,096 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_037a/L1.47.4.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.4.las'
2017-01-19 17:12:30,096 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_037a/L1.47.5.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.5.las'
2017-01-19 17:12:30,096 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_037a/L1.47.6.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.6.las'
2017-01-19 17:12:30,097 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_037a/L1.47.7.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.7.las'
2017-01-19 17:12:30,097 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02a7/L1.47.20.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.20.las'
2017-01-19 17:12:30,097 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02a7/L1.47.21.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.21.las'
2017-01-19 17:12:30,097 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02a7/L1.47.22.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.22.las'
2017-01-19 17:12:30,097 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02a7/L1.47.23.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.23.las'
2017-01-19 17:12:30,097 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02a1/L1.47.83.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.83.las'
2017-01-19 17:12:30,098 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02a2/L1.47.58.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.58.las'
2017-01-19 17:12:30,098 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02ae/L1.47.52.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.52.las'
2017-01-19 17:12:30,098 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_011c/L1.47.93.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.93.las'
2017-01-19 17:12:30,098 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_005f/L1.47.16.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.16.las'
2017-01-19 17:12:30,098 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_005f/L1.47.17.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.17.las'
2017-01-19 17:12:30,098 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_005f/L1.47.18.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.18.las'
2017-01-19 17:12:30,099 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_005f/L1.47.19.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.19.las'
2017-01-19 17:12:30,099 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_005b/L1.47.44.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.44.las'
2017-01-19 17:12:30,099 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_005b/L1.47.45.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.45.las'
2017-01-19 17:12:30,099 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_005b/L1.47.46.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.46.las'
2017-01-19 17:12:30,099 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_005b/L1.47.47.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.47.las'
2017-01-19 17:12:30,099 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0113/L1.47.68.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.68.las'
2017-01-19 17:12:30,099 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_046d/L1.47.49.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.49.las'
2017-01-19 17:12:30,100 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0467/L1.47.95.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.95.las'
2017-01-19 17:12:30,100 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0464/L1.47.96.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.96.las'
2017-01-19 17:12:30,100 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03b0/L1.47.67.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.67.las'
2017-01-19 17:12:30,100 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03e9/L1.47.60.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.60.las'
2017-01-19 17:12:30,100 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_00cc/L1.47.70.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.70.las'
2017-01-19 17:12:30,100 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01c1/L1.47.1.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.1.las'
2017-01-19 17:12:30,101 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01c1/L1.47.2.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.2.las'
2017-01-19 17:12:30,101 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01c1/L1.47.3.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.3.las'
2017-01-19 17:12:30,101 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01d3/L1.47.63.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.63.las'
2017-01-19 17:12:30,101 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01db/L1.47.12.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.12.las'
2017-01-19 17:12:30,101 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01db/L1.47.13.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.13.las'
2017-01-19 17:12:30,101 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01db/L1.47.14.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.14.las'
2017-01-19 17:12:30,102 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01db/L1.47.15.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.15.las'
2017-01-19 17:12:30,102 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01dd/L1.47.57.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.57.las'
2017-01-19 17:12:30,102 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_012e/L1.47.24.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.24.las'
2017-01-19 17:12:30,102 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_012e/L1.47.25.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.25.las'
2017-01-19 17:12:30,102 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_012e/L1.47.26.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.26.las'
2017-01-19 17:12:30,102 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_012e/L1.47.27.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.27.las'
2017-01-19 17:12:30,102 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01e7/L1.47.64.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.64.las'
2017-01-19 17:12:30,103 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0319/L1.47.73.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.73.las'
2017-01-19 17:12:30,103 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0323/L1.47.61.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.61.las'
2017-01-19 17:12:30,103 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0326/L1.47.48.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.48.las'
2017-01-19 17:12:30,103 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0285/L1.47.40.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.40.las'
2017-01-19 17:12:30,103 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0285/L1.47.41.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.41.las'
2017-01-19 17:12:30,103 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0285/L1.47.42.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.42.las'
2017-01-19 17:12:30,104 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0285/L1.47.43.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.43.las'
2017-01-19 17:12:30,104 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0280/L1.47.65.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.65.las'
2017-01-19 17:12:30,104 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_003b/L1.47.51.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.51.las'
2017-01-19 17:12:30,104 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_032f/L1.47.98.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.98.las'
2017-01-19 17:12:30,104 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0033/L1.47.59.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.59.las'
2017-01-19 17:12:30,104 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0032/L1.47.77.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.77.las'
2017-01-19 17:12:30,105 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_028f/L1.47.81.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.81.las'
2017-01-19 17:12:30,105 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02b6/L1.47.88.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.88.las'
2017-01-19 17:12:30,105 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0386/L1.47.75.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.75.las'
2017-01-19 17:12:30,105 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0158/L1.47.55.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.55.las'
2017-01-19 17:12:30,105 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_04e1/L1.47.84.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.84.las'
2017-01-19 17:12:30,105 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_04e9/L1.47.32.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.32.las'
2017-01-19 17:12:30,105 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_04e9/L1.47.33.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.33.las'
2017-01-19 17:12:30,106 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_04e9/L1.47.34.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.34.las'
2017-01-19 17:12:30,106 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_04e9/L1.47.35.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.35.las'
2017-01-19 17:12:30,106 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0498/L1.47.94.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.94.las'
2017-01-19 17:12:30,106 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03f8/L1.47.53.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.53.las'
2017-01-19 17:12:30,106 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0010/L1.47.82.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.82.las'
2017-01-19 17:12:30,106 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0080/L1.47.89.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.89.las'
2017-01-19 17:12:30,107 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03fa/L1.47.66.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.66.las'
2017-01-19 17:12:30,107 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_04dd/L1.47.74.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.74.las'
2017-01-19 17:12:30,107 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_04cd/L1.47.72.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.72.las'
2017-01-19 17:12:30,107 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0493/L1.47.90.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.90.las'
2017-01-19 17:12:30,107 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0006/L1.47.71.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.71.las'
2017-01-19 17:12:30,107 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0000/L1.47.76.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.76.las'
2017-01-19 17:12:30,108 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_0073/L1.47.91.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.91.las'
2017-01-19 17:12:30,108 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_049f/L1.47.97.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.97.las'
2017-01-19 17:12:30,108 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_01aa/L1.47.78.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.78.las'
2017-01-19 17:12:30,108 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_04bb/L1.47.79.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.79.las'
2017-01-19 17:12:30,108 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02fd/L1.47.36.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.36.las'
2017-01-19 17:12:30,108 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02fd/L1.47.37.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.37.las'
2017-01-19 17:12:30,108 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02fd/L1.47.38.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.38.las'
2017-01-19 17:12:30,109 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_02fd/L1.47.39.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.39.las'
2017-01-19 17:12:30,109 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_03c5/L1.47.99.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.99.las'
2017-01-19 17:12:30,109 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_024c/L1.47.50.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.50.las'
2017-01-19 17:12:30,109 - falcon_kit.pype_tasks - DEBUG - symlink '../../../../../../../../../../../../0-rawreads/job_008b/L1.47.92.las' -> '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/L1.47.92.las'
2017-01-19 17:12:30,110 - pypeflow.do_support - INFO - !/bin/bash -vex /Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/rp_00047.sh
#!/bin/bash
set -vex
+ set -vex

# Get the real path of a link, following all links (since 'readlink -f' is not on OSX).
# http://stackoverflow.com/questions/7665/how-to-resolve-symbolic-links-in-a-shell-script/697552
realpath() {
    local r=$1; local t=$(readlink $r)
    while [ $t ]; do
        r=$(cd $(dirname $r) && cd $(dirname $t) && pwd -P)/$(basename $t)
        t=$(readlink $r)
    done
    echo $r
}
rmfollow() {
    local path=$(realpath $1)
    rm -f "${path}"
}
LAmerge -v raw_reads.47 L1.47.1 L1.47.2 L1.47.3 L1.47.4 L1.47.5 L1.47.6 L1.47.7 L1.47.8 L1.47.9 L1.47.10 L1.47.11 L1.47.12 L1.47.13 L1.47.14 L1.47.15 L1.47.16 L1.47.17 L1.47.18 L1.47.19 L1.47.20 L1.47.21 L1.47.22 L1.47.23 L1.47.24 L1.47.25 L1.47.26 L1.47.27 L1.47.28 L1.47.29 L1.47.30 L1.47.31 L1.47.32 L1.47.33 L1.47.34 L1.47.35 L1.47.36 L1.47.37 L1.47.38 L1.47.39 L1.47.40 L1.47.41 L1.47.42 L1.47.43 L1.47.44 L1.47.45 L1.47.46 L1.47.47 L1.47.48 L1.47.49 L1.47.50 L1.47.51 L1.47.52 L1.47.53 L1.47.54 L1.47.55 L1.47.56 L1.47.57 L1.47.58 L1.47.59 L1.47.60 L1.47.61 L1.47.62 L1.47.63 L1.47.64 L1.47.65 L1.47.66 L1.47.67 L1.47.68 L1.47.69 L1.47.70 L1.47.71 L1.47.72 L1.47.73 L1.47.74 L1.47.75 L1.47.76 L1.47.77 L1.47.78 L1.47.79 L1.47.80 L1.47.81 L1.47.82 L1.47.83 L1.47.84 L1.47.85 L1.47.86 L1.47.87 L1.47.88 L1.47.89 L1.47.90 L1.47.91 L1.47.92 L1.47.93 L1.47.94 L1.47.95 L1.47.96 L1.47.97 L1.47.98 L1.47.99 L1.47.100
+ LAmerge -v raw_reads.47 L1.47.1 L1.47.2 L1.47.3 L1.47.4 L1.47.5 L1.47.6 L1.47.7 L1.47.8 L1.47.9 L1.47.10 L1.47.11 L1.47.12 L1.47.13 L1.47.14 L1.47.15 L1.47.16 L1.47.17 L1.47.18 L1.47.19 L1.47.20 L1.47.21 L1.47.22 L1.47.23 L1.47.24 L1.47.25 L1.47.26 L1.47.27 L1.47.28 L1.47.29 L1.47.30 L1.47.31 L1.47.32 L1.47.33 L1.47.34 L1.47.35 L1.47.36 L1.47.37 L1.47.38 L1.47.39 L1.47.40 L1.47.41 L1.47.42 L1.47.43 L1.47.44 L1.47.45 L1.47.46 L1.47.47 L1.47.48 L1.47.49 L1.47.50 L1.47.51 L1.47.52 L1.47.53 L1.47.54 L1.47.55 L1.47.56 L1.47.57 L1.47.58 L1.47.59 L1.47.60 L1.47.61 L1.47.62 L1.47.63 L1.47.64 L1.47.65 L1.47.66 L1.47.67 L1.47.68 L1.47.69 L1.47.70 L1.47.71 L1.47.72 L1.47.73 L1.47.74 L1.47.75 L1.47.76 L1.47.77 L1.47.78 L1.47.79 L1.47.80 L1.47.81 L1.47.82 L1.47.83 L1.47.84 L1.47.85 L1.47.86 L1.47.87 L1.47.88 L1.47.89 L1.47.90 L1.47.91 L1.47.92 L1.47.93 L1.47.94 L1.47.95 L1.47.96 L1.47.97 L1.47.98 L1.47.99 L1.47.100
LAmerge: Cannot open ./L1.47.100.las for 'r'
2017-01-19 17:12:31,135 - root - DEBUG - CD: '/Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp//rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047' -> '/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047'
2017-01-19 17:12:31,136 - root - DEBUG - CD: '/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047' -> '/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047'
2017-01-19 17:12:31,136 - root - CRITICAL - Error in /Project/Genome_Assembly/PacBio/Tools/Falcon/FALCON-integrate/pypeFLOW/pypeflow/do_task.py with args="{'json_fn': '/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/task.json',\n 'timeout': 60,\n 'tmpdir': '/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/tmp'}"
Traceback (most recent call last):
  File "/Project/Genome_Assembly/PacBio/Tools/Python-2.7.12/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/Project/Genome_Assembly/PacBio/Tools/Python-2.7.12/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Project/Genome_Assembly/PacBio/Tools/Falcon/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 190, in <module>
    main()
  File "/Project/Genome_Assembly/PacBio/Tools/Falcon/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 182, in main
    run(**vars(parsed_args))
  File "/Project/Genome_Assembly/PacBio/Tools/Falcon/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 136, in run
    run_cfg_in_tmpdir(cfg, tmpdir)
  File "/Project/Genome_Assembly/PacBio/Tools/Falcon/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 160, in run_cfg_in_tmpdir
    run_python_func(func, myinputs, myoutputs, parameters)
  File "/Project/Genome_Assembly/PacBio/Tools/Falcon/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 125, in run_python_func
    do_support.run_bash(script_fn)
  File "/Project/Genome_Assembly/PacBio/Tools/Falcon/FALCON-integrate/pypeFLOW/pypeflow/do_support.py", line 51, in run_bash
    raise Exception('{} <- {!r}'.format(rc, cmd))
Exception: 256 <- '/bin/bash -vex /Project/Genome_Assembly/PacBio/Stage_1/tmp/pypetmp/rd1/user/Project/Genome_Assembly/PacBio/Stage_1/0-rawreads/m_00047/rp_00047.sh'

real    0m1.137s
user    0m0.079s
sys     0m0.036s
 returned: 256
pb-cdunn commented 7 years ago

I further "rm 0-rawreads/m && rm mypwatcher" and restart fcrun.py, now all new "0-rawreads/m job were failed for lacking of *las files.

That's because the rmfollow lines in the generated bash scripts for merge will delete the original .las files via symbolic link.

What is the right way to restart my job?

With the default pwatcher, just rm -rf mypwatcher

In fact, las files such as L1.71.1.las could be found in 0-rawreads/job_*, why it complain couldn't found such files?

I don't know yet. You'll have to step into a directory like 0-rawreads/m_00047 and investigate at the command-line in your shell.

Which version of FALCON-integrate are you using? Or which commits of FALCON and pypeFLOW?

lamz138138 commented 7 years ago

Hi, thanks for your reply!

  1. The version of FALCON-integrate is "a72f47c4600fb31c993151c1d03c1787ba42b161", FALCON is "7a6ac0d8e8492c64733a997d72a9359e1275bb57", and pypeFLOW is "f928267429f8cb456191b518a6d0de966b772ddc".

  2. In fact, after "rm -rf mypwatcher" and restart fc_run.py with , it exit about a minute later. After checking the m_00047 in the tmp director, I think the job failed because it didn't make link for "L1.47.100.las" (I had use L1.71.1 for m_00090 previously). Relate information been updated above. Any suggestion would be grateful! 

    fc_run.py fc_run.cfg >fc_run.log 2>fc_run.err
grep "failed" fc_run.err 
[ERROR]Task Node(0-rawreads/m_00047) failed with exit-code=256
[ERROR]Task Node(0-rawreads/m_00090) failed with exit-code=256
dgordon562 commented 7 years ago

I've had many of these errors in the past.

LAmerge: Cannot open ./L1.47.100.las

check if L1.47.100.las exists. It should be there and be a link to one of the job_ directories. I'll bet it isn't there.

Then you need to try to figure out which job directory it should have pointed to. When you figure that out, then look in the job directory and see if L1.47.100.las exists. I'll bet it doesn't. (This is the way it always worked for me).

So you will need to re-run daligner on that job directory. If you don't want to do that manually, you can get Falcon to do it by deleting the done flag for that job directory. I think there may be another done flag to delete as well (which indicates all job directories have been processed). Chris can fill in the gaps in my instructions.

The reason this occurs is bad error-handling in daligner. If daligner can't, for example, write a file, it will not crash but rather continue to run. Thus Falcon will have no signal that daligner has failed and will write the done flag. I would love it if Gene/Chris could add some error handling in daligner so it will raise an error if it can't write a file or a line.