Closed agroppi closed 6 years ago
To be more specific here is the stderr file in /mypwatcher/heartbeats/heartbeat-P19e9459cfd107e
+
python2.7 -m pwatcher.mains.fs_heartbeat --directory=/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/3-unzip/reads --heartbeat-file=/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/mypwatcher/heartbeats/heartbeat-P19e9459cfd107e --exit-file=/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/mypwatcher/exits/exit-P19e9459cfd107e --rate=10.0 /bin/bash run.sh
Namespace(command=['/bin/bash', 'run.sh'], directory='/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/3-unzip/reads', exit_file='/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/mypwatcher/exits/exit-P19e9459cfd107e', heartbeat_file='/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/mypwatcher/heartbeats/heartbeat-P19e9459cfd107e', rate=10.0)
cwd:'/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/3-unzip/reads'
hostname=cortex02.cluster
heartbeat_fn='/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/mypwatcher/heartbeats/heartbeat-P19e9459cfd107e'
exit_fn='/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/mypwatcher/exits/exit-P19e9459cfd107e'
sleep_s=10.0
before setpgid: pid=100286 pgid=100188
after setpgid: pid=100286 pgid=100286
In cwd: /gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/3-unzip/reads, Blocking call: '/bin/bash run.sh'
cd /gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/3-unzip/reads
+
cd /gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/3-unzip/reads
bash task.sh
+
bash task.sh
Eureka ! I found this old issue #81
You have to change the name of your fofn file into input.fofn. Unzip only supports this name so fat. It didn't parse the name in your unzip.cfg.
So far so good, unzip seems to run
solved
I used install_unzip.sh in http://pb-falcon.readthedocs.io/en/latest/quick_start.html to install falcon and falcon-unzip.
falcon-kit==0.7+git.2059148090374ac08a494d842dc1def105aeee50
This step worked perfectly The FALCON assembly worked also very well. But now I'm facing an issue with FALCON_unzip :2017-12-01 11:27:48,752 - pwatcher.fs_based - DEBUG - Unable to remove heartbeat '/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/mypwatcher/heartbeats/heartbeat-P19e9459cfd107e' when sentinal was found in exit-sentinels listdir. Traceback (most recent call last): File "/module/apps/falcon/0.7/fc_env/lib/python2.7/site-packages/pwatcher/fs_based.py", line 565, in get_status os.remove(heartbeat_path) OSError: [Errno 2] No such file or directory: '/gpfs/scratch/ag/ABXING/Falcon_0.7_Mandshurica_3kb/mypwatcher/heartbeats/heartbeat-P19e9459cfd107e' 2017-12-01 11:27:48,753 - pwatcher.fs_based - DEBUG - Status EXIT 256 for heartbeat:heartbeat-P19e9459cfd107e 2017-12-01 11:27:48,753 - pypeflow.simple_pwatcher_bridge - ERROR - Task Node(3-unzip/reads) failed with exit-code=256 2017-12-01 11:27:48,753 - pypeflow.simple_pwatcher_bridge - DEBUG - recently_done: [(Node(3-unzip/reads), False)] 2017-12-01 11:27:48,753 - pypeflow.simple_pwatcher_bridge - DEBUG - Num done in this iteration: 1 2017-12-01 11:27:48,753 - pypeflow.simple_pwatcher_bridge - INFO - recently_satisfied: set([]) 2017-12-01 11:27:48,753 - pypeflow.simple_pwatcher_bridge - INFO - Num satisfied in this iteration: 0 2017-12-01 11:27:48,753 - pypeflow.simple_pwatcher_bridge - INFO - Num still unsatisfied: 1 2017-12-01 11:27:48,753 - pypeflow.simple_pwatcher_bridge - ERROR - Some tasks are recently_done but not satisfied: set([Node(3-unzip/reads)]) 2017-12-01 11:27:48,753 - pypeflow.simple_pwatcher_bridge - ERROR - ready: set([]) submitted: set([]) 2017-12-01 11:27:48,753 - pwatcher.fs_based - DEBUG - delete(which='known', jobids=<0>) 2017-12-01 11:27:48,753 - pwatcher.fs_based - DEBUG - Deleting jobs for jobids from known ([]) 2017-12-01 11:27:48,754 - pwatcher.fs_based - DEBUG - Failed to kill job for heartbeat 'heartbeat-P19e9459cfd107e': IOError(2, 'No such file or directory') 2017-12-01 11:27:48,754 - pwatcher.fs_based - DEBUG - Cannot remove heartbeat: OSError(2, 'No such file or directory') 2017-12-01 11:27:48,754 - pypeflow.simple_pwatcher_bridge - DEBUG - In notifyTerminate(), result of delete:None
I attach here the output/error and all.log files Falcon_0.7_Unzip_Mandshurica_3kb_56807.out.zip all.log.zip