Is it right to have track_reads_done.exit and OSError in all.log?

[lamz138138]

Hi, I had run falcon_unzip with Arabidopsis data, although it finished with status of "done", I found something indicating that there maybe some problem of this process, so I ask for help here, any suggestion would be grateful!

1. I found there is a file called "track_reads_done.exit" in 3-unzip/reads/, I think it should only contain track_reads_done, so is it OK to have this file? And I found two errors in 3-unzip/reads/pwatcher.dir/stderr:

   
   head 3-unzip/reads/pwatcher.dir/stderr 
   bash: module: line 1: syntax error: unexpected end of file
   bash: error importing function definition for `module'
   + python2.7 -m pwatcher.mains.fs_heartbeat --directory=/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/3-unzip/reads --heartbeat-file=/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/heartbeats/heartbeat-P5548614fd6fb82 --exit-file=/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/exits/exit-P5548614fd6fb82 --rate=10.0 /bin/bash run.sh
   Namespace(command=['/bin/bash', 'run.sh'], directory='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/3-unzip/reads', exit_file='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/exits/exit-P5548614fd6fb82', heartbeat_file='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/heartbeats/heartbeat-P5548614fd6fb82', rate=10.0)

cwd:'/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/3-unzip/reads' hostname=smp01.local heartbeat_fn='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/heartbeats/heartbeat-P5548614fd6fb82' exit_fn='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/exits/exit-P5548614fd6fb82' sleep_s=10.0

Is this a problem? And whether the problem is caused by lacking of "#!/bin/bash" in the first line of  track_reads.sh, and how can I add this line to track_reads.sh?

cat 3-unzip/reads/track_reads.sh set -vex trap 'touch /data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/3-unzip/reads/track_reads_done.exit' EXIT hostname date cd ../.. python -m falcon_kit.mains.get_read_ctg_map python -m falcon_kit.mains.rr_ctg_track python -m falcon_kit.mains.pr_ctg_track

mkdir -p 3-unzip/reads/

python -m falcon_kit.mains.fetch_reads cd /data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/3-unzip/reads date touch /data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/3-unzip/reads/track_reads_done

Besides, I had found follow error in "all.log", but I can't figure out when it happened, and which script lead to it, is this a problem?

2017-11-04 14:43:35,576 - pwatcher.fs_based - DEBUG - Status RUNNING for heartbeat:heartbeat-Pc4e6f6f1ee39b9 2017-11-04 14:43:35,576 - pwatcher.fs_based - DEBUG - Status RUNNING for heartbeat:heartbeat-P5f6ca498daff0a 2017-11-04 14:43:35,576 - pwatcher.fs_based - DEBUG - Status RUNNING for heartbeat:heartbeat-Pf9b50f328566b0 2017-11-04 14:43:35,576 - pwatcher.fs_based - DEBUG - Status RUNNING for heartbeat:heartbeat-Pf90a85ff9e4dde 2017-11-04 14:43:35,576 - pypeflow.simple_pwatcher_bridge - INFO - sleep 1.6s 2017-11-04 14:43:37,178 - pypeflow.simple_pwatcher_bridge - DEBUG - N in queue: 4 (max_jobs=8) 2017-11-04 14:43:37,179 - pwatcher.fs_based - DEBUG - query(which='list', jobids=<4>) 2017-11-04 14:43:37,182 - pwatcher.fs_based - DEBUG - Unable to remove heartbeat '/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/heartbeats/heartbeat-Pc4e6f6f1ee39b9' when sentinal was found in exit-sentinels listdir. Traceback (most recent call last): File "/data7/lcy/zhongxm/tools/Falcon/Falcon_unzip/myVirtualenv/lib/python2.7/site-packages/pwatcher/fs_based.py", line 565, in get_status os.remove(heartbeat_path) OSError: [Errno 2] No such file or directory: '/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/heartbeats/heartbeat-Pc4e6f6f1ee39b9'

2017-11-04 14:43:37,183 - pwatcher.fs_based - DEBUG - Status EXIT 0 for heartbeat:heartbeat-Pc4e6f6f1ee39b9 2017-11-04 14:43:37,183 - pwatcher.fs_based - DEBUG - Status RUNNING for heartbeat:heartbeat-P5f6ca498daff0a 2017-11-04 14:43:37,183 - pwatcher.fs_based - DEBUG - Status RUNNING for heartbeat:heartbeat-Pf9b50f328566b0 2017-11-04 14:43:37,183 - pwatcher.fs_based - DEBUG - Status RUNNING for heartbeat:heartbeat-Pf90a85ff9e4dde 2017-11-04 14:43:37,183 - pypeflow.simple_pwatcher_bridge - DEBUG - recently_done: [(Node(3-unzip/0-phasing/000012F/phasing), True)] 2017-11-04 14:43:37,184 - pypeflow.simple_pwatcher_bridge - DEBUG - Num done in this iteration: 1 2017-11-04 14:43:37,184 - pypeflow.simple_pwatcher_bridge - INFO - recently_satisfied: set([Node(3-unzip/0-phasing/000012F/phasing)]) 2017-11-04 14:43:37,184 - pypeflow.simple_pwatcher_bridge - INFO - Num satisfied in this iteration: 1

[lamz138138]

I had saw related issue said the problem of heartbeat is fine

edit: After running quiver, I found "track_reads_h_done.exit" in 4-quiver/reads, and the file of "cns_p_ctg.fasta/q" is empty. Also found follow information in 4-quiver/reads/pwatcher.dir/stderr:

head 4-quiver/reads/pwatcher.dir/stderr 
bash: module: line 1: syntax error: unexpected end of file
bash: error importing function definition for `module'
+ python2.7 -m pwatcher.mains.fs_heartbeat --directory=/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/4-quiver/reads --heartbeat-file=/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/heartbeats/heartbeat-Pa6e8f6d091b16a --exit-file=/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/exits/exit-Pa6e8f6d091b16a --rate=10.0 /bin/bash run.sh
Namespace(command=['/bin/bash', 'run.sh'], directory='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/4-quiver/reads', exit_file='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/exits/exit-Pa6e8f6d091b16a', heartbeat_file='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/heartbeats/heartbeat-Pa6e8f6d091b16a', rate=10.0)

cwd:'/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/4-quiver/reads'
hostname=smp03.local
heartbeat_fn='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/heartbeats/heartbeat-Pa6e8f6d091b16a'
exit_fn='/data12/lcy01/zhongxm/PacBio/falcon_unzip/demo/mypwatcher/exits/exit-Pa6e8f6d091b16a'
sleep_s=10.0