FALCON Unzip ERROR during first step in "reads" folder

[uceleste]

Hi All,

I'm trying to run Falcon Unzip on my already obtained Falcon assembly. This is the error I obtain when I run the run.sh:

sh run.sh
cd /home/data/pisa_ngs/Backup_Genetics/PacBio/Fig_Falcon8e/3-unzip/reads
+ cd /home/data/pisa_ngs/Backup_Genetics/PacBio/Fig_Falcon8e/3-unzip/reads
bash task.sh
+ bash task.sh
: command not foundstname
: command not foundrt
: command not foundd
2018-05-10 08:28:42,919 - root - DEBUG - Running "/home/data/bioinf_resources/programming_tools/falcontest/lib/python2.7/site-packages/pypeflow-1.0.0-py2.7.egg/pypeflow"do_task.py --tmpdir /das_data /home/data/pisa_ngs/Backup_Genetics/PacBio/Fig_Falcon8e/3-unzip/reads/task.json
2018-05-10 08:28:42,938 - root - DEBUG - Checking existence of '/home/data/pisa_ngs/Backup_Genetics/PacBio/Fig_Falcon8e/3-unzip/reads/task.json\r' with timeout=60
2018-05-10 08:29:43,241 - root - CRITICAL - Error in /home/data/bioinf_resources/programming_tools/falcontest/lib/python2.7/site-packages/pypeflow-1.0.0-py2.7.egg/pypeflow/do_task.py with args="{'json_fn': '/home/data/pisa_ngs/Backup_Genetics/PacBio/Fig_Falcon8e/3-unzip/reads/task.json\\r',\n 'timeout': 60,\n 'tmpdir': '/das_data'}"
Traceback (most recent call last):
  File "/home/apps/python/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/home/apps/python/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/data/bioinf_resources/programming_tools/falcontest/lib/python2.7/site-packages/pypeflow-1.0.0-py2.7.egg/pypeflow/do_task.py", line 217, in <module>
    main()
  File "/home/data/bioinf_resources/programming_tools/falcontest/lib/python2.7/site-packages/pypeflow-1.0.0-py2.7.egg/pypeflow/do_task.py", line 209, in main
    run(**vars(parsed_args))
  File "/home/data/bioinf_resources/programming_tools/falcontest/lib/python2.7/site-packages/pypeflow-1.0.0-py2.7.egg/pypeflow/do_task.py", line 197, in run
    wait_for(json_fn)
  File "/home/data/bioinf_resources/programming_tools/falcontest/lib/python2.7/site-packages/pypeflow-1.0.0-py2.7.egg/pypeflow/do_task.py", line 69, in wait_for
    raise Exception('Timed out waiting for {!r}'.format(fn))
Exception: Timed out waiting for '/home/data/pisa_ngs/Backup_Genetics/PacBio/Fig_Falcon8e/3-unzip/reads/task.json\r'

real    1m9.517s
user    0m0.208s
sys     0m0.692s

This is the ending part of the log file:

2018-05-09 22:01:20,865 - pwatcher.fs_based - DEBUG - query(which='list', jobids=<1>)
2018-05-09 22:01:20,868 - pwatcher.fs_based - DEBUG - Status RUNNING for heartbeat:heartbeat-P1541c59517c0f5
2018-05-09 22:01:20,868 - pypeflow.simple_pwatcher_bridge - INFO - sleep 10s
2018-05-09 22:01:30,878 - pypeflow.simple_pwatcher_bridge - DEBUG - N in queue: 1 (max_jobs=8)
2018-05-09 22:01:30,879 - pwatcher.fs_based - DEBUG - query(which='list', jobids=<1>)
2018-05-09 22:01:30,881 - pwatcher.fs_based - DEBUG - Status RUNNING for heartbeat:heartbeat-P1541c59517c0f5
2018-05-09 22:01:30,881 - pypeflow.simple_pwatcher_bridge - INFO - sleep 10s
2018-05-09 22:01:40,891 - pypeflow.simple_pwatcher_bridge - DEBUG - N in queue: 1 (max_jobs=8)
2018-05-09 22:01:40,892 - pwatcher.fs_based - DEBUG - query(which='list', jobids=<1>)
2018-05-09 22:01:41,091 - pwatcher.fs_based - DEBUG - Unable to remove heartbeat '/home/data/pisa_ngs/Backup_Genetics/PacBio/Fig_Falcon8e/mypwatcher/heartbeats/heartbeat-P1541c59517c0f5' when sentinal was found in exit-sentinels listdir.
Traceback (most recent call last):
  File "/home/data/bioinf_resources/programming_tools/falcontest/lib/python2.7/site-packages/pypeflow-1.0.0-py2.7.egg/pwatcher/fs_based.py", line 565, in get_status
    os.remove(heartbeat_path)
OSError: [Errno 2] No such file or directory: '/home/data/pisa_ngs/Backup_Genetics/PacBio/Fig_Falcon8e/mypwatcher/heartbeats/heartbeat-P1541c59517c0f5'

2018-05-09 22:01:41,109 - pwatcher.fs_based - DEBUG - Status EXIT 32512 for heartbeat:heartbeat-P1541c59517c0f5
2018-05-09 22:01:41,110 - pypeflow.simple_pwatcher_bridge - ERROR - Task Node(3-unzip/reads) failed with exit-code=32512
2018-05-09 22:01:41,110 - pypeflow.simple_pwatcher_bridge - DEBUG - recently_done: [(Node(3-unzip/reads), False)]
2018-05-09 22:01:41,110 - pypeflow.simple_pwatcher_bridge - DEBUG - Num done in this iteration: 1
2018-05-09 22:01:41,110 - pypeflow.simple_pwatcher_bridge - INFO - recently_satisfied: set([])
2018-05-09 22:01:41,111 - pypeflow.simple_pwatcher_bridge - INFO - Num satisfied in this iteration: 0
2018-05-09 22:01:41,111 - pypeflow.simple_pwatcher_bridge - INFO - Num still unsatisfied: 1
2018-05-09 22:01:41,111 - pypeflow.simple_pwatcher_bridge - ERROR - Some tasks are recently_done but not satisfied: set([Node(3-unzip/reads)])
2018-05-09 22:01:41,111 - pypeflow.simple_pwatcher_bridge - ERROR - ready: set([])
    submitted: set([])
2018-05-09 22:01:41,112 - pwatcher.fs_based - DEBUG - delete(which='known', jobids=<0>)
2018-05-09 22:01:41,112 - pwatcher.fs_based - DEBUG - Deleting jobs for jobids from known ([])
2018-05-09 22:01:41,114 - pwatcher.fs_based - DEBUG - Failed to kill job for heartbeat 'heartbeat-P1541c59517c0f5': IOError(2, 'No such file or directory')
2018-05-09 22:01:41,116 - pwatcher.fs_based - DEBUG - Cannot remove heartbeat: OSError(2, 'No such file or directory')
2018-05-09 22:01:41,117 - pypeflow.simple_pwatcher_bridge - DEBUG - In notifyTerminate(), result of delete:None

Thank you in advance for any advice.

[pb-cdunn]

: command not foundstname
: command not foundrt
: command not foundd

We've never seen this and probably cannot help you from here. stname problem comes from hostname. rt might come from sort, and d from pwd. In each case, you are getting only the first two characters. Could this be a wide-character problem? Totally guessing.

My task.sh looks like this:

$ cat 0-rawreads/build/task.sh
#!/bin/bash
onerror () {
  set -vx
  echo "FAILURE. Running top in $(pwd) (If you see -terminal database is inaccessible- you are using the python bin-wrapper, so you will not get diagnostic info. No big deal. This process is crashing anyway.)"
  rm -f top.txt
  which python
  which top
  env -u LD_LIBRARY_PATH top -b -n 1 >| top.txt &
  env -u LD_LIBRARY_PATH top -b -n 1 2>&1
  pstree -apl
}
trap onerror ERR
env | sort

echo "HOSTNAME=$(hostname)"
echo "PWD=$(pwd)"

time python2.7 -m pypeflow.do_task  /localdisk/scratch/cdunn/repo/FALCON-examples/run/synth0/0-rawreads/build/task.json

Try the python command directly. If that works, then there is a problem with your bash -- maybe an old version of bash? If you solve this, please let us know what you did or learned.