Peregrine not recognising options

[ghost]

Hello,

I am using Peregrine on my computer, here is my commande

sudo docker run -it -v /mnt/sda1/Alessandro/07-12-2019_Peregrine/ cschin/peregrine:0.1.5.5 asm \
    /mnt/sda1/Alessandro/07-12-2019_Peregrine/reads.lst 24 24 24 24 24 24 24 24 24 \
    --with-consensus --shimmer-r 3 --best_n_ovlp 8 \
    --output /mnt/sda1/Alessandro/07-12-2019_Peregrine/Peregrine_output

And the output I get

Peregrine Assembler (0+unknown) has been started with the following option:
 {'--aln_bw': '100',
 '--best_n_ovlp': '4',
 '--help': False,
 '--mc_lower': '2',
 '--mc_upper': '240',
 '--min_idt': '96',
 '--min_len': '4000',
 '--output': './wd',
 '--ovlp_upper': '120',
 '--shimmer-k': '16',
 '--shimmer-l': '2',
 '--shimmer-r': '6',
 '--shimmer-w': '80',
 '--verison': False,
 '--with-L0-index': False,
 '--with-consensus': False,
 '<cns_nchunk>': '24',
 '<cns_nproc>': '24',
 '<index_nchunk>': '24',
 '<index_nproc>': '24',
 '<mapping_nchunk>': '24',
 '<mapping_nproc>': '24',
 '<ovlp_nchunk>': '24',
 '<ovlp_nproc>': '24',
 '<reads.lst>': '/mnt/sda1/Alessandro/07-12-2019_Peregrine/reads.lst',
 '<sort_nproc>': '24',
 'asm': True}
INFO:pypeflow.simple_pwatcher_bridge:In simple_pwatcher_bridge, pwatcher_impl=<module 'pwatcher.blocking' from '/opt/conda/envs/peregrine/lib/python3.7/site-packages/pypeflow-2.1.1+git.d63b0e79f5a7b2d370b
7de84a890f88271afa476-py3.7.egg/pwatcher/blocking.py'>
INFO:pypeflow.simple_pwatcher_bridge:job_type='local', (default)job_defaults={'njobs': 1, 'NPROC': 1, 'MB': 24000, 'submit': 'bash -C ${CMD} >| ${STDOUT_FILE} 2>| ${STDERR_FILE}', 'job_type': 'local', 'pw
atcher_type': 'blocking'}, use_tmpdir=None, squash=False, job_name_style=0
INFO:pypeflow.simple_pwatcher_bridge:Setting max_jobs to 1; was None
INFO:pypeflow.simple_pwatcher_bridge:Num unsatisfied: 1, graph: 1
INFO:pypeflow.simple_pwatcher_bridge:About to submit: Node(wd/0-seqdb)
INFO:pwatcher.blocking:Popen: '/bin/bash -C /opt/conda/envs/peregrine/lib/python3.7/site-packages/pypeflow-2.1.1+git.d63b0e79f5a7b2d370b7de84a890f88271afa476-py3.7.egg/pwatcher/mains/job_start.sh >| /opt/
test/wd/0-seqdb/run-P5a619370806c87.bash.stdout 2>| /opt/test/wd/0-seqdb/run-P5a619370806c87.bash.stderr'
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 0.0s -- another 1 loop iterations)
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 0.30000000000000004s -- another 2 loop iterations)
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 1.2000000000000002s -- another 3 loop iterations)
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 2.9999999999999996s -- another 4 loop iterations)

INFO:pypeflow.simple_pwatcher_bridge:(slept for another 6.0s -- another 5 loop iterations)
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 10.500000000000002s -- another 6 loop iterations)
ERROR:pypeflow.simple_pwatcher_bridge:Task Node(wd/0-seqdb) failed with exit-code=1
ERROR:pypeflow.simple_pwatcher_bridge:Some tasks are recently_done but not satisfied: {Node(wd/0-seqdb)}
ERROR:pypeflow.simple_pwatcher_bridge:ready: set()
        submitted: set()
ERROR:pwatcher.blocking:Noop. We cannot kill blocked threads. Hopefully, everything will die on SIGTERM.
Traceback (most recent call last):
  File "/opt/conda/envs/peregrine/bin/pg_run.py", line 4, in <module>
    __import__('pkg_resources').run_script('peregrine==0+unknown', 'pg_run.py')
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/pkg_resources/__init__.py", line 666, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1462, in run_script
    exec(code, namespace, namespace)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/peregrine-0+unknown-py3.7-linux-x86_64.egg/EGG-INFO/scripts/pg_run.py", line 658, in <module>
    main(args)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/peregrine-0+unknown-py3.7-linux-x86_64.egg/EGG-INFO/scripts/pg_run.py", line 573, in main
    read_db_abs_prefix, read_db = run_build_db(wf, args, seq_dataset_lst)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/peregrine-0+unknown-py3.7-linux-x86_64.egg/EGG-INFO/scripts/pg_run.py", line 220, in run_build_db
    wf.refreshTargets()
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/pypeflow-2.1.1+git.d63b0e79f5a7b2d370b7de84a890f88271afa476-py3.7.egg/pypeflow/simple_pwatcher_bridge.py", line 278, in refreshTargets
    self._refreshTargets(updateFreq, exitOnFailure)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/pypeflow-2.1.1+git.d63b0e79f5a7b2d370b7de84a890f88271afa476-py3.7.egg/pypeflow/simple_pwatcher_bridge.py", line 362, in _refreshTargets
    raise Exception(msg)
Exception: Some tasks are recently_done but not satisfied: {Node(wd/0-seqdb)}
zsh: command not found: --with-consensus
zsh: command not found: --output

It doesn't seem to recognise the options. What am I doing wrong? Thank you

[cschin]

you might have unseen tailing space after this line

 /mnt/sda1/Alessandro/07-12-2019_Peregrine/reads.lst 24 24 24 24 24 24 24 24 24 \

Try to submit the job using a single long line to see if that is the case.

[ghost]

Hello, I have done it, it removes the zsh command not found error but it is still not working

sudo docker run -it -v /mnt/sda1/Alessandro/07-12-2019_Peregrine/ cschin/peregrine:0.1.5.5 asm /mnt/sda1/Alessandro/07-12-2019_Peregrine/reads.lst 24 24 24 24 24 24 24 24 2 --with-consensus --shimmer-r 3 --best_n_ovlp 8 --output /mnt/sda1/Alessandro/07-12-2019_Peregrine/Peregrine_output

INFO:pypeflow.simple_pwatcher_bridge:Setting max_jobs to 1; was None
INFO:pypeflow.simple_pwatcher_bridge:Num unsatisfied: 1, graph: 1
INFO:pypeflow.simple_pwatcher_bridge:About to submit: Node(../../mnt/sda1/Alessandro/07-12-2019_Peregrine/Peregrine_output/0-seqdb)                                                                        
INFO:pwatcher.blocking:Popen: '/bin/bash -C /opt/conda/envs/peregrine/lib/python3.7/site-packages/pypeflow-2.1.1+git.d63b0e79f5a7b2d370b7de84a890f88271afa476-py3.7.egg/pwatcher/mains/job_start.sh >| /mnt/sda1/Alessandro/07-12-2019_Peregrine/Peregrine_output/0-seqdb/run-P5b1d3a0c093c95.bash.stdout 2>| /mnt/sda1/Alessandro/07-12-2019_Peregrine/Peregrine_output/0-seqdb/run-P5b1d3a0c093c95.bash.stderr'      
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 0.0s -- another 1 loop iterations)
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 0.30000000000000004s -- another 2 loop iterations)                                                                                                 
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 1.2000000000000002s -- another 3 loop iterations)                                                                                                  
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 2.9999999999999996s -- another 4 loop iterations)                                                                                                  
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 6.0s -- another 5 loop iterations)
INFO:pypeflow.simple_pwatcher_bridge:(slept for another 10.500000000000002s -- another 6 loop iterations)                                                                                                  
ERROR:pypeflow.simple_pwatcher_bridge:Task Node(../../mnt/sda1/Alessandro/07-12-2019_Peregrine/Peregrine_output/0-seqdb) failed with exit-code=1                                                           
ERROR:pypeflow.simple_pwatcher_bridge:Some tasks are recently_done but not satisfied: {Node(../../mnt/sda1/Alessandro/07-12-2019_Peregrine/Peregrine_output/0-seqdb)}                                      
ERROR:pypeflow.simple_pwatcher_bridge:ready: set()
        submitted: set()
ERROR:pwatcher.blocking:Noop. We cannot kill blocked threads. Hopefully, everything will die on SIGTERM.                                                                                                   
Traceback (most recent call last):
  File "/opt/conda/envs/peregrine/bin/pg_run.py", line 4, in <module>
    __import__('pkg_resources').run_script('peregrine==0+unknown', 'pg_run.py')
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/pkg_resources/__init__.py", line 666, in run_script                                                                                          
    self.require(requires)[0].run_script(script_name, ns)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1462, in run_script                                                                                         
    exec(code, namespace, namespace)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/peregrine-0+unknown-py3.7-linux-x86_64.egg/EGG-INFO/scripts/pg_run.py", line 658, in <module>                                                
    main(args)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/peregrine-0+unknown-py3.7-linux-x86_64.egg/EGG-INFO/scripts/pg_run.py", line 573, in main                                                    
    read_db_abs_prefix, read_db = run_build_db(wf, args, seq_dataset_lst)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/peregrine-0+unknown-py3.7-linux-x86_64.egg/EGG-INFO/scripts/pg_run.py", line 220, in run_build_db                                            
    wf.refreshTargets()
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/pypeflow-2.1.1+git.d63b0e79f5a7b2d370b7de84a890f88271afa476-py3.7.egg/pypeflow/simple_pwatcher_bridge.py", line 278, in refreshTargets       
    self._refreshTargets(updateFreq, exitOnFailure)
  File "/opt/conda/envs/peregrine/lib/python3.7/site-packages/pypeflow-2.1.1+git.d63b0e79f5a7b2d370b7de84a890f88271afa476-py3.7.egg/pypeflow/simple_pwatcher_bridge.py", line 362, in _refreshTargets      
    raise Exception(msg)
Exception: Some tasks are recently_done but not satisfied: {Node(../../mnt/sda1/Alessandro/07-12-2019_Peregrine/Peregrine_output/0-seqdb)}  

I am running Ubuntu 18.04 and Docker 19.03.5 if that helps

[cschin]

This is because the code can not find the sequence files, please check the file paths inside /mnt/sda1/Alessandro/07-12-2019_Peregrine/reads.lst

[ghost]

the path is correct as far as I can tell

cat reads.lst 
/mnt/sda1/Alessandro/07-12-2019_Peregrine/ontvaga_45kb.oneline.fa

pwd
/mnt/sda1/Alessandro/07-12-2019_Peregrine

ls
CONSENT  ontvaga_45kb.fa  ontvaga_45kb.oneline.fa  reads.lst

[cschin]

How much memory do you have and what is the size of the reads (in terms of sequencing data)?

[ghost]

Hello, I have 250 Go of memory and the reads are >= 40 Kb.

[cschin]

Is this 40kb accurate reads? (>99% accuracy)