wdecoster / nanocomp

Comparison of multiple long read datasets
MIT License
113 stars 9 forks source link

Error running NanoComp #66

Closed ekushele closed 1 year ago

ekushele commented 1 year ago

Hello,

I'm trying to run NanoComp, and I'm getting the following error:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/NanoComp", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/nanocomp/NanoComp.py", line 52, in main
    datadf = nanoget.get_input(
  File "/usr/local/lib/python3.10/site-packages/nanoget/nanoget.py", line 110, in get_input
    dfs=[out for out in executor.map(extraction_function, files)],
  File "/usr/local/lib/python3.10/site-packages/nanoget/nanoget.py", line 110, in <listcomp>
    dfs=[out for out in executor.map(extraction_function, files)],
  File "/usr/local/lib/python3.10/concurrent/futures/process.py", line 570, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
slurmstepd: error: Detected 17 oom-kill event(s) in StepId=17246554.batch cgroup. Some of your processes may have been killed by the cgroup out-of-memory handler.
wdecoster commented 1 year ago

While that is not the full error message, based on the line below it seems you are running against the cgroup memory settings. You will have to discuss that with your system administrator.

Some of your processes may have been killed by the cgroup out-of-memory handler.

ekushele commented 1 year ago

Thank you!