search

nf-core / pangenome

Renders a collection of sequences into a pangenome graph.
https://nf-co.re/pangenome
MIT License
63 stars 16 forks source link

Issues with odgibuild #42

Closed dbrami closed 3 years ago

dbrami commented 3 years ago

This can't be a memory issue as I have 64GB on my machine and only adding 2 x 200 viral genomes

Description of the bug

odgibuild chokes but running the following completes succesfully:

nextflow run nf-core/pangenome -r dev -profile test,docker

Steps to reproduce

Steps to reproduce the behaviour:

export NXF_OPTS="-Xms8G -Xmx60G"
nextflow run nf-core/pangenome -r dev -profile docker --input 'ncbi200_??.fasta.gz' -c nfcore_ec2.config
N E X T F L O W  ~  version 20.10.0
Launching `nf-core/pangenome` [pensive_galileo] - revision: 2652c2871a [dev]
----------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/pangenome v1.0dev
----------------------------------------------------
executor >  local (15)
[72/e5d415] process > edyeet (2)    [100%] 2 of 2 ✔
[d6/66334d] process > seqwish (2)   [100%] 2 of 2 ✔
[11/d228c2] process > smoothxg (2)  [100%] 2 of 2 ✔
[e6/be2b2c] process > odgiBuild (7) [  0%] 0 of 11
[-        ] process > odgiStats     -
[-        ] process > odgiViz       -
[-        ] process > odgiChop      -
[-        ] process > odgiLayout    -
[-        ] process > odgiDraw      -
Error executing process > 'odgiBuild (1)'

Caused by:
  Process `odgiBuild (1)` terminated with an error exit status (1)

Command executed:

  odgi build -g ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa -o ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa.og

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  Passed in argument, but no positional arguments were ready to receive it: ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa
executor >  local (15)
[72/e5d415] process > edyeet (2)    [100%] 2 of 2 ✔
[d6/66334d] process > seqwish (2)   [100%] 2 of 2 ✔
[11/d228c2] process > smoothxg (2)  [100%] 2 of 2 ✔
[6f/1a2ca0] process > odgiBuild (2) [ 18%] 2 of 11, failed: 1
[-        ] process > odgiStats     -
[-        ] process > odgiViz       -
[-        ] process > odgiChop      -
[-        ] process > odgiLayout    -
[-        ] process > odgiDraw      -
Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'odgiBuild (1)'

Caused by:
  Process `odgiBuild (1)` terminated with an error exit status (1)

Command executed:

  odgi build -g ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa -o ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa.og

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  Passed in argument, but no positional arguments were ready to receive it: ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa
executor >  local (15)
[72/e5d415] process > edyeet (2)    [100%] 2 of 2 ✔
[d6/66334d] process > seqwish (2)   [100%] 2 of 2 ✔
[11/d228c2] process > smoothxg (2)  [100%] 2 of 2 ✔
[42/1ed10c] process > odgiBuild (6) [ 36%] 4 of 11, failed: 1
[-        ] process > odgiStats     -
[-        ] process > odgiViz       -
[-        ] process > odgiChop      -
[-        ] process > odgiLayout    -
[-        ] process > odgiDraw      -
Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'odgiBuild (1)'

Caused by:
  Process `odgiBuild (1)` terminated with an error exit status (1)

Command executed:

  odgi build -g ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa -o ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa.og

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  Passed in argument, but no positional arguments were ready to receive it: ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa
executor >  local (15)
[72/e5d415] process > edyeet (2)    [100%] 2 of 2 ✔
[d6/66334d] process > seqwish (2)   [100%] 2 of 2 ✔
[11/d228c2] process > smoothxg (2)  [100%] 2 of 2 ✔
[f5/ff32be] process > odgiBuild (5) [ 64%] 7 of 11, failed: 1
[-        ] process > odgiStats     -
[-        ] process > odgiViz       -
[-        ] process > odgiChop      -
[-        ] process > odgiLayout    -
[-        ] process > odgiDraw      -
Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'odgiBuild (1)'

Caused by:
  Process `odgiBuild (1)` terminated with an error exit status (1)

Command executed:

  odgi build -g ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa -o ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa.og

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  Passed in argument, but no positional arguments were ready to receive it: ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa
executor >  local (15)
[72/e5d415] process > edyeet (2)    [100%] 2 of 2 ✔
[d6/66334d] process > seqwish (2)   [100%] 2 of 2 ✔
[11/d228c2] process > smoothxg (2)  [100%] 2 of 2 ✔
[0f/6f0f89] process > odgiBuild (9) [ 73%] 8 of 11, failed: 1
[-        ] process > odgiStats     -
[-        ] process > odgiViz       -
[-        ] process > odgiChop      -
[-        ] process > odgiLayout    -
[-        ] process > odgiDraw      -
Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'odgiBuild (1)'

Caused by:
  Process `odgiBuild (1)` terminated with an error exit status (1)

Command executed:

  odgi build -g ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa -o ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa.og

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  Passed in argument, but no positional arguments were ready to receive it: ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa
executor >  local (15)
[72/e5d415] process > edyeet (2)    [100%] 2 of 2 ✔
[d6/66334d] process > seqwish (2)   [100%] 2 of 2 ✔
[11/d228c2] process > smoothxg (2)  [100%] 2 of 2 ✔
[e6/be2b2c] process > odgiBuild (7) [ 82%] 9 of 11, failed: 1
[-        ] process > odgiStats     -
[-        ] process > odgiViz       -
[-        ] process > odgiChop      -
[-        ] process > odgiLayout    -
[-        ] process > odgiDraw      -
Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'odgiBuild (1)'

Caused by:
  Process `odgiBuild (1)` terminated with an error exit status (1)

Command executed:

  odgi build -g ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa -o ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa.og

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  Passed in argument, but no positional arguments were ready to receive it: ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa
    odgi build {OPTIONS}
executor >  local (15)
[72/e5d415] process > edyeet (2)    [100%] 2 of 2 ✔
[d6/66334d] process > seqwish (2)   [100%] 2 of 2 ✔
[11/d228c2] process > smoothxg (2)  [100%] 2 of 2 ✔
[e6/be2b2c] process > odgiBuild (7) [ 82%] 9 of 11, failed: 1
[-        ] process > odgiStats     -
[-        ] process > odgiViz       -
[-        ] process > odgiChop      -
[-        ] process > odgiLayout    -
[-        ] process > odgiDraw      -
Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'odgiBuild (1)'

Caused by:
  Process `odgiBuild (1)` terminated with an error exit status (1)

Command executed:

  odgi build -g ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa -o ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa.og

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  Passed in argument, but no positional arguments were ready to receive it: ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa
    odgi build {OPTIONS}

      construct a dynamic succinct variation graph

    OPTIONS:

        -h, --help                        display this help summary
        -g[FILE], --gfa=[FILE]            construct the graph from this GFA input
                                          file
        -o[FILE], --out=[FILE]            store the graph self index in this file
        -G, --to-gfa                      write the graph to stdout in GFA format
        -s, --sort                        apply generalized topological sort to
                                          the graph and set node ids to order
        -t[N], --threads=[N]              number of threads to use for parallel
                                          operations
        -d, --debug                       enable debugging
        -p, --progress                    show progress updates

Work dir:
  /home/ubuntu/covid19/1k/work/64/97145e147f3682ed595a0fbb7125bb

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
One more CTRL+C to force exit

Expected behaviour

completion of odgibuild step

Log files

Have you provided the following extra information/files:

Feb-21 17:19:59.658 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: seqwish, odgiStats, odgiChop, odgiViz, odgiDraw, smoothxg, edyeet, odgiLayout, odgiBuild Feb-21 17:19:59.706 [main] DEBUG nextflow.script.ProcessConfig - Config settings withName:edyeet matches process edyeet Feb-21 17:19:59.711 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Feb-21 17:19:59.711 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Feb-21 17:19:59.715 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local Feb-21 17:19:59.719 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=32; memory=62.1 GB; capacity=32; pollInterval=100ms; dumpInterval=5m Feb-21 17:19:59.778 [main] DEBUG nextflow.script.ProcessConfig - Config settings withName:seqwish matches process seqwish Feb-21 17:19:59.779 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Feb-21 17:19:59.779 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Feb-21 17:19:59.784 [main] DEBUG nextflow.script.ProcessConfig - Config settings withName:smoothxg matches process smoothxg Feb-21 17:19:59.785 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Feb-21 17:19:59.785 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Feb-21 17:19:59.802 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Feb-21 17:19:59.802 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Feb-21 17:19:59.810 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Feb-21 17:19:59.810 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Feb-21 17:19:59.824 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Feb-21 17:19:59.824 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Feb-21 17:19:59.832 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Feb-21 17:19:59.832 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Feb-21 17:19:59.837 [main] DEBUG nextflow.script.ProcessConfig - Config settings withName:odgiLayout matches process odgiLayout Feb-21 17:19:59.838 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Feb-21 17:19:59.838 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Feb-21 17:19:59.843 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null Feb-21 17:19:59.843 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local' Feb-21 17:19:59.846 [main] DEBUG nextflow.Session - Ignite dataflow network (11) Feb-21 17:19:59.852 [PathVisitor-1] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: ./; pattern: ncbi200_??.fasta.gz; options: [:] Feb-21 17:19:59.852 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > edyeet Feb-21 17:19:59.853 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > seqwish Feb-21 17:19:59.853 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > smoothxg Feb-21 17:19:59.854 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > odgiBuild Feb-21 17:19:59.854 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > odgiStats Feb-21 17:19:59.855 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > odgiViz Feb-21 17:19:59.856 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > odgiChop Feb-21 17:19:59.859 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > odgiLayout Feb-21 17:19:59.860 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > odgiDraw Feb-21 17:19:59.861 [main] DEBUG nextflow.script.ScriptRunner - > Await termination Feb-21 17:19:59.861 [main] DEBUG nextflow.Session - Session await Feb-21 17:20:00.058 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:00.063 [Task submitter] INFO nextflow.Session - [4e/c0b0b0] Submitted process > edyeet (1) Feb-21 17:20:02.128 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: edyeet (1); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/4e/c0b0b06122abc248f0df55ac374e07] Feb-21 17:20:02.135 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:02.136 [Task submitter] INFO nextflow.Session - [72/e5d415] Submitted process > edyeet (2) Feb-21 17:20:04.248 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 2; name: edyeet (2); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/72/e5d415e32467b49cf0f2ff7a4257f2] Feb-21 17:20:04.254 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:04.255 [Task submitter] INFO nextflow.Session - [91/1b9515] Submitted process > seqwish (1) Feb-21 17:20:07.566 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 3; name: seqwish (1); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/91/1b9515299b6cac1ccdaf53aefabc98] Feb-21 17:20:07.573 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:07.573 [Task submitter] INFO nextflow.Session - [d6/66334d] Submitted process > seqwish (2) Feb-21 17:20:11.045 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: seqwish (2); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/d6/66334dfe99c0e515877d0d9104e3a0] Feb-21 17:20:11.049 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:11.051 [Task submitter] INFO nextflow.Session - [5a/22ef1f] Submitted process > smoothxg (1) Feb-21 17:20:24.281 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 5; name: smoothxg (1); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/5a/22ef1fe3d9a99c09b3f1313062a791] Feb-21 17:20:24.284 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:24.285 [Task submitter] INFO nextflow.Session - [11/d228c2] Submitted process > smoothxg (2) Feb-21 17:20:37.238 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 6; name: smoothxg (2); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/11/d228c2551fa6f50fe3cbfde6f2be2a] Feb-21 17:20:37.244 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:37.244 [Task submitter] INFO nextflow.Session - [64/97145e] Submitted process > odgiBuild (1) Feb-21 17:20:37.251 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:37.251 [Task submitter] INFO nextflow.Session - [6f/1a2ca0] Submitted process > odgiBuild (2) Feb-21 17:20:37.257 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:37.258 [Task submitter] INFO nextflow.Session - [35/c9ffaa] Submitted process > odgiBuild (3) Feb-21 17:20:37.264 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:37.265 [Task submitter] INFO nextflow.Session - [00/2e50e7] Submitted process > odgiBuild (4) Feb-21 17:20:37.270 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:37.270 [Task submitter] INFO nextflow.Session - [f5/ff32be] Submitted process > odgiBuild (5) Feb-21 17:20:37.278 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:37.278 [Task submitter] INFO nextflow.Session - [42/1ed10c] Submitted process > odgiBuild (6) Feb-21 17:20:37.285 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:37.286 [Task submitter] INFO nextflow.Session - [93/c1fbcb] Submitted process > odgiBuild (8) Feb-21 17:20:37.291 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:37.292 [Task submitter] INFO nextflow.Session - [0f/6f0f89] Submitted process > odgiBuild (9) Feb-21 17:20:38.736 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 7; name: odgiBuild (1); status: COMPLETED; exit: 1; error: -; workDir: /home/ubuntu/covid19/1k/work/64/97145e147f3682ed595a0fbb7125bb] Feb-21 17:20:38.740 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run Feb-21 17:20:38.740 [Task submitter] INFO nextflow.Session - [e6/be2b2c] Submitted process > odgiBuild (7) Feb-21 17:20:38.756 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'odgiBuild (1)'

Caused by: Process odgiBuild (1) terminated with an error exit status (1)

Command executed:

odgi build -g ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa -o ncbi200_01.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa.og

Command exit status: 1

Command output: (empty)

Command error: WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap. Passed in argument, but no positional arguments were ready to receive it: ncbi200_00.pggb-s1000-p90-n10-a90-K16-X-k8-w10000-j5000-W0-e5000-I0.seqwish.gfa odgi build {OPTIONS}

  construct a dynamic succinct variation graph

OPTIONS:

    -h, --help                        display this help summary
    -g[FILE], --gfa=[FILE]            construct the graph from this GFA input
                                      file
    -o[FILE], --out=[FILE]            store the graph self index in this file
    -G, --to-gfa                      write the graph to stdout in GFA format
    -s, --sort                        apply generalized topological sort to
                                      the graph and set node ids to order
    -t[N], --threads=[N]              number of threads to use for parallel
                                      operations
    -d, --debug                       enable debugging
    -p, --progress                    show progress updates

Work dir: /home/ubuntu/covid19/1k/work/64/97145e147f3682ed595a0fbb7125bb

Tip: when you have fixed the problem you can continue the execution adding the option -resume to the run command line Feb-21 17:20:38.762 [Task monitor] INFO nextflow.Session - Execution cancelled -- Finishing pending tasks before exit Feb-21 17:20:38.767 [main] DEBUG nextflow.Session - Session await > all process finished Feb-21 17:20:38.917 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 8; name: odgiBuild (2); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/6f/1a2ca03c5f86cf3f7e80194141be8d] Feb-21 17:20:39.133 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 9; name: odgiBuild (3); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/35/c9ffaa3083edc3b337ba216e245cff] Feb-21 17:20:39.143 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 12; name: odgiBuild (6); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/42/1ed10c7dc13a281644c45ee02875a9] Feb-21 17:20:39.218 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 10; name: odgiBuild (4); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/00/2e50e72f8cea4e5d1bee56fb00f7e4] Feb-21 17:20:39.284 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 14; name: odgiBuild (8); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/93/c1fbcb70eb3a257c5c5bbed9c53350] Feb-21 17:20:39.348 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 11; name: odgiBuild (5); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/f5/ff32beec67cafb605bc4962960ab33] Feb-21 17:20:39.415 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 15; name: odgiBuild (9); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/0f/6f0f89881f72465dc241363cc8efa3] Feb-21 17:20:40.183 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 13; name: odgiBuild (7); status: COMPLETED; exit: 0; error: -; workDir: /home/ubuntu/covid19/1k/work/e6/be2b2cfc860739a0569bc5747d427f] Feb-21 17:20:53.935 [SIGINT handler] DEBUG nextflow.Session - Session aborted -- Cause: SIGINT Feb-21 17:20:53.948 [main] DEBUG nextflow.Session - Session await > all barriers passed Feb-21 17:20:53.955 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=14; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=2; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=17m 45s; failedDuration=1.4s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=8; peakCpus=32; peakMemory=56 GB; ] Feb-21 17:20:53.955 [main] DEBUG nextflow.trace.TraceFileObserver - Flow completing -- flushing trace file Feb-21 17:20:53.957 [main] DEBUG nextflow.trace.ReportObserver - Flow completing -- rendering html report Feb-21 17:20:53.995 [main] DEBUG nextflow.trace.ReportObserver - Execution report summary data: [{"cpuUsage":{"mean":55.11,"min":54.18,"q1":54.65,"q2":55.11,"q3":55.57,"max":56.04,"minLabel":"edyeet (1)","maxLabel":"edyeet (2)","q1Label":"edyeet (1)","q2Label":"edyeet (1)","q3Label":"edyeet (1)"},"process":"edyeet","mem":{"mean":69402624,"min":62636032,"q1":66019328,"q2":69402624,"q3":72785920,"max":76169216,"minLabel":"edyeet (1)","maxLabel":"edyeet (2)","q1Label":"edyeet (1)","q2Label":"edyeet (1)","q3Label":"edyeet (1)"},"memUsage":{"mean":0.92,"min":0.83,"q1":0.88,"q2":0.92,"q3":0.97,"max":1.01,"minLabel":"edyeet (1)","maxLabel":"edyeet (2)","q1Label":"edyeet (1)","q2Label":"edyeet (1)","q3Label":"edyeet (1)"},"timeUsage":{"mean":0.01,"min":0.01,"q1":0.01,"q2":0.01,"q3":0.01,"max":0.01,"minLabel":"edyeet (1)","maxLabel":"edyeet (2)","q1Label":"edyeet (1)","q2Label":"edyeet (1)","q3Label":"edyeet (1)"},"vmem":{"mean":2748823552,"min":2664906752,"q1":2706865152,"q2":2748823552,"q3":2790781952,"max":2832740352,"minLabel":"edyeet (1)","maxLabel":"edyeet (2)","q1Label":"edyeet (1)","q2Label":"edyeet (1)","q3Label":"edyeet (1)"},"reads":{"mean":8531422,"min":8288934,"q1":8410178,"q2":8531422,"q3":8652666,"max":8773910,"minLabel":"edyeet (1)","maxLabel":"edyeet (2)","q1Label":"edyeet (1)","q2Label":"edyeet (1)","q3Label":"edyeet (1)"},"cpu":{"mean":1763.55,"min":1733.9,"q1":1748.73,"q2":1763.55,"q3":1778.38,"max":1793.2,"minLabel":"edyeet (1)","maxLabel":"edyeet (2)","q1Label":"edyeet (1)","q2Label":"edyeet (1)","q3Label":"edyeet (1)"},"time":{"mean":1432.5,"min":1427,"q1":1429.75,"q2":1432.5,"q3":1435.25,"max":1438,"minLabel":"edyeet (1)","maxLabel":"edyeet (2)","q1Label":"edyeet (1)","q2Label":"edyeet (1)","q3Label":"edyeet (1)"},"writes":{"mean":9861364.5,"min":9152765,"q1":9507064.75,"q2":9861364.5,"q3":10215664.25,"max":10569964,"minLabel":"edyeet (1)","maxLabel":"edyeet (2)","q1Label":"edyeet (1)","q2Label":"edyeet (1)","q3Label":"edyeet (1)"}},{"cpuUsage":{"mean":13.63,"min":13.54,"q1":13.59,"q2":13.63,"q3":13.67,"max":13.71,"minLabel":"seqwish (2)","maxLabel":"seqwish (1)","q1Label":"seqwish (2)","q2Label":"seqwish (2)","q3Label":"seqwish (2)"},"process":"seqwish","mem":{"mean":309073920,"min":308404224,"q1":308739072,"q2":309073920,"q3":309408768,"max":309743616,"minLabel":"seqwish (1)","maxLabel":"seqwish (2)","q1Label":"seqwish (1)","q2Label":"seqwish (1)","q3Label":"seqwish (1)"},"memUsage":{"mean":4.11,"min":4.1,"q1":4.11,"q2":4.11,"q3":4.12,"max":4.12,"minLabel":"seqwish (1)","maxLabel":"seqwish (2)","q1Label":"seqwish (1)","q2Label":"seqwish (1)","q3Label":"seqwish (1)"},"timeUsage":{"mean":0.02,"min":0.02,"q1":0.02,"q2":0.02,"q3":0.02,"max":0.02,"minLabel":"seqwish (1)","maxLabel":"seqwish (2)","q1Label":"seqwish (1)","q2Label":"seqwish (1)","q3Label":"seqwish (1)"},"vmem":{"mean":2790244352,"min":2789535744,"q1":2789890048,"q2":2790244352,"q3":2790598656,"max":2790952960,"minLabel":"seqwish (1)","maxLabel":"seqwish (2)","q1Label":"seqwish (1)","q2Label":"seqwish (1)","q3Label":"seqwish (1)"},"reads":{"mean":8925805.5,"min":8342255,"q1":8634030.25,"q2":8925805.5,"q3":9217580.75,"max":9509356,"minLabel":"seqwish (1)","maxLabel":"seqwish (2)","q1Label":"seqwish (1)","q2Label":"seqwish (1)","q3Label":"seqwish (1)"},"cpu":{"mean":436.1,"min":433.4,"q1":434.75,"q2":436.1,"q3":437.45,"max":438.8,"minLabel":"seqwish (2)","maxLabel":"seqwish (1)","q1Label":"seqwish (2)","q2Label":"seqwish (2)","q3Label":"seqwish (2)"},"time":{"mean":2734.5,"min":2627,"q1":2680.75,"q2":2734.5,"q3":2788.25,"max":2842,"minLabel":"seqwish (1)","maxLabel":"seqwish (2)","q1Label":"seqwish (1)","q2Label":"seqwish (1)","q3Label":"seqwish (1)"},"writes":{"mean":22461965.5,"min":21945142,"q1":22203553.75,"q2":22461965.5,"q3":22720377.25,"max":22978789,"minLabel":"seqwish (2)","maxLabel":"seqwish (1)","q1Label":"seqwish (2)","q2Label":"seqwish (2)","q3Label":"seqwish (2)"}},{"cpuUsage":{"mean":45.86,"min":45.71,"q1":45.79,"q2":45.86,"q3":45.94,"max":46.01,"minLabel":"smoothxg (2)","maxLabel":"smoothxg (1)","q1Label":"smoothxg (2)","q2Label":"smoothxg (2)","q3Label":"smoothxg (2)"},"process":"smoothxg","mem":{"mean":158607360,"min":157413376,"q1":158010368,"q2":158607360,"q3":159204352,"max":159801344,"minLabel":"smoothxg (2)","maxLabel":"smoothxg (1)","q1Label":"smoothxg (2)","q2Label":"smoothxg (2)","q3Label":"smoothxg (2)"},"memUsage":{"mean":2.11,"min":2.09,"q1":2.1,"q2":2.11,"q3":2.12,"max":2.13,"minLabel":"smoothxg (2)","maxLabel":"smoothxg (1)","q1Label":"smoothxg (2)","q2Label":"smoothxg (2)","q3Label":"smoothxg (2)"},"timeUsage":{"mean":0.09,"min":0.09,"q1":0.09,"q2":0.09,"q3":0.09,"max":0.09,"minLabel":"smoothxg (2)","maxLabel":"smoothxg (1)","q1Label":"smoothxg (2)","q2Label":"smoothxg (2)","q3Label":"smoothxg (2)"},"vmem":{"mean":4910694400,"min":4893085696,"q1":4901890048,"q2":4910694400,"q3":4919498752,"max":4928303104,"minLabel":"smoothxg (1)","maxLabel":"smoothxg (2)","q1Label":"smoothxg (1)","q2Label":"smoothxg (1)","q3Label":"smoothxg (1)"},"reads":{"mean":683634,"min":680499,"q1":682066.5,"q2":683634,"q3":685201.5,"max":686769,"minLabel":"smoothxg (1)","maxLabel":"smoothxg (2)","q1Label":"smoothxg (1)","q2Label":"smoothxg (1)","q3Label":"smoothxg (1)"},"cpu":{"mean":1467.6,"min":1462.8,"q1":1465.2,"q2":1467.6,"q3":1470,"max":1472.4,"minLabel":"smoothxg (2)","maxLabel":"smoothxg (1)","q1Label":"smoothxg (2)","q2Label":"smoothxg (2)","q3Label":"smoothxg (2)"},"time":{"mean":12464,"min":12321,"q1":12392.5,"q2":12464,"q3":12535.5,"max":12607,"minLabel":"smoothxg (2)","maxLabel":"smoothxg (1)","q1Label":"smoothxg (2)","q2Label":"smoothxg (2)","q3Label":"smoothxg (2)"},"writes":{"mean":166984954.5,"min":159681754,"q1":163333354.25,"q2":166984954.5,"q3":170636554.75,"max":174288155,"minLabel":"smoothxg (2)","maxLabel":"smoothxg (1)","q1Label":"smoothxg (2)","q2Label":"smoothxg (2)","q3Label":"smoothxg (2)"}},{"cpuUsage":{"mean":91.21,"min":78.7,"q1":86.13,"q2":91.85,"q3":94.9,"max":102.4,"minLabel":"odgiBuild (3)","maxLabel":"odgiBuild (8)","q1Label":"odgiBuild (4)","q2Label":"odgiBuild (6)","q3Label":"odgiBuild (2)"},"process":"odgiBuild","mem":{"mean":9841664,"min":7434240,"q1":9765888,"q2":9863168,"q3":10107904,"max":11767808,"minLabel":"odgiBuild (2)","maxLabel":"odgiBuild (7)","q1Label":"odgiBuild (9)","q2Label":"odgiBuild (3)","q3Label":"odgiBuild (4)"},"memUsage":{"mean":0.13,"min":0.1,"q1":0.13,"q2":0.13,"q3":0.13,"max":0.16,"minLabel":"odgiBuild (2)","maxLabel":"odgiBuild (7)","q1Label":"odgiBuild (9)","q2Label":"odgiBuild (3)","q3Label":"odgiBuild (4)"},"timeUsage":{"mean":0,"min":0,"q1":0,"q2":0,"q3":0,"max":0.01,"minLabel":"odgiBuild (6)","maxLabel":"odgiBuild (1)","q1Label":"odgiBuild (4)","q2Label":"odgiBuild (8)","q3Label":"odgiBuild (7)"},"vmem":{"mean":17021952,"min":15572992,"q1":16760832,"q2":16766976,"q3":16990208,"max":19230720,"minLabel":"odgiBuild (2)","maxLabel":"odgiBuild (7)","q1Label":"odgiBuild (6)","q2Label":"odgiBuild (9)","q3Label":"odgiBuild (3)"},"reads":{"mean":48518.38,"min":48509,"q1":48516.5,"q2":48520,"q3":48522,"max":48524,"minLabel":"odgiBuild (2)","maxLabel":"odgiBuild (6)","q1Label":"odgiBuild (7)","q2Label":"odgiBuild (8)","q3Label":"odgiBuild (4)"},"cpu":{"mean":91.21,"min":78.7,"q1":86.13,"q2":91.85,"q3":94.9,"max":102.4,"minLabel":"odgiBuild (3)","maxLabel":"odgiBuild (8)","q1Label":"odgiBuild (4)","q2Label":"odgiBuild (6)","q3Label":"odgiBuild (2)"},"time":{"mean":236.78,"min":28,"q1":29,"q2":33,"q3":234,"max":1398,"minLabel":"odgiBuild (6)","maxLabel":"odgiBuild (1)","q1Label":"odgiBuild (4)","q2Label":"odgiBuild (8)","q3Label":"odgiBuild (7)"},"writes":{"mean":1794090.63,"min":778618,"q1":784742,"q2":845022.5,"q3":1792265.75,"max":4960143,"minLabel":"odgiBuild (6)","maxLabel":"odgiBuild (2)","q1Label":"odgiBuild (5)","q2Label":"odgiBuild (4)","q3Label":"odgiBuild (3)"}}] Feb-21 17:20:54.722 [main] DEBUG nextflow.trace.TimelineObserver - Flow completing -- rendering html timeline Feb-21 17:20:54.760 [main] DEBUG nextflow.CacheDB - Closing CacheDB done Feb-21 17:20:54.763 [main] DEBUG nextflow.util.SpuriousDeps - AWS S3 uploader shutdown Feb-21 17:20:54.767 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye



## System

- Hardware: AWS c5a.8xlarge
- Executor: local
- OS: ubuntu 
- Version 20.04

## Nextflow Installation

- Version: 20.10

## Container engine

- Engine: <Docker
- version: Docker version 19.03.13, build cd8016b6bc
- Image tag: nfcore/pangenome:dev

## Additional context

<!-- Add any other context about the problem here. -->
subwaystation commented 3 years ago

Hi @dbrami I suspect you ran into the same issues as in #41 and #39 .

dbrami commented 3 years ago

Thank you Simon. I found the requirement to have a wildcard character within the command line for the input file name misleading as I specifically split up my input multi-fasta into multiple files to comply with the wildcard requirement...

On Mon, Feb 22, 2021 at 12:38 PM Simon Heumos notifications@github.com wrote:

Hi @dbrami https://github.com/dbrami I suspect you ran into the same issues as in #41 https://github.com/nf-core/pangenome/issues/41 and #39 https://github.com/nf-core/pangenome/issues/39 .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nf-core/pangenome/issues/42#issuecomment-783238640, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABH7WZEOHZRWLDXVK4PRUTTAIQ3HANCNFSM4X7GOBZQ .

subwaystation commented 3 years ago

Oh. Now I understand the confusion! Sorry Daniel. I will update the README to reflect the what pangenome is capable of.

subwaystation commented 3 years ago

This is already tracked in #39