search

XPRESSyourself / XPRESSpipe

An alignment and analysis pipeline for Ribosome Profiling and RNA-seq data
https://xpresspipe.readthedocs.io/en/latest/
GNU General Public License v3.0
13 stars 4 forks source link

Metagene error #24

Closed j-berg closed 5 years ago

j-berg commented 5 years ago 
  File "/uufs/chpc.utah.edu/common/home/u0690617/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1122, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/uufs/chpc.utah.edu/common/home/u0690617/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1853, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 387, in pandas._libs.parsers.TextReader.__cinit__
  File "pandas/_libs/parsers.pyx", line 705, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: [Errno 2] File b'/scratch/general/lustre/u0690617/7559123/output/metagene/metrics/SRR1795418_1_Aligned.metaposit' does not exist: b'/scratch/gen
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/uufs/chpc.utah.edu/common/home/u0690617/miniconda3/bin/xpresspipe", line 11, in <module>
    load_entry_point('XPRESSpipe', 'console_scripts', 'xpresspipe')()
  File "/uufs/chpc.utah.edu/common/home/u0690617/XPRESSpipe/xpresspipe/__main__.py", line 257, in main
    make_metagene(args_dict, files)
  File "/uufs/chpc.utah.edu/common/home/u0690617/XPRESSpipe/xpresspipe/metagene.py", line 178, in make_metagene
    mod_workers = True)
  File "/uufs/chpc.utah.edu/common/home/u0690617/XPRESSpipe/xpresspipe/parallel.py", line 158, in parallelize
    args_dict)
  File "/uufs/chpc.utah.edu/common/home/u0690617/XPRESSpipe/xpresspipe/parallel.py", line 124, in run_pools
    for file in zip(batch, executor.map(func, batch)):
  File "/uufs/chpc.utah.edu/common/home/u0690617/miniconda3/lib/python3.7/concurrent/futures/process.py", line 476, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/uufs/chpc.utah.edu/common/home/u0690617/miniconda3/lib/python3.7/concurrent/futures/_base.py", line 586, in result_iterator
    yield fs.pop().result()
  File "/uufs/chpc.utah.edu/common/home/u0690617/miniconda3/lib/python3.7/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/uufs/chpc.utah.edu/common/home/u0690617/miniconda3/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
FileNotFoundError: [Errno 2] File b'/scratch/general/lustre/u0690617/7559123/output/metagene/metrics/SRR1795418_1_Aligned.metaposit' does not exist: b'/scratch/gen
Command exited with non-zero status 1
        Command being timed: "xpresspipe metagene -i /scratch/general/lustre/u0690617/7559123/output/alignments_transcriptome -o /scratch/general/lustre/u0690617/7
        User time (seconds): 4570.53
        System time (seconds): 734.95
        Percent of CPU this job got: 171%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 51:36.22
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 20399072
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 554232
        Minor (reclaiming a frame) page faults: 279087144
        Voluntary context switches: 868524
        Involuntary context switches: 3128485
        Swaps: 0
        File system inputs: 107859984
        File system outputs: 11793168
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 1
j-berg commented 5 years ago

Not sure it gives a FileNotFoundError, but its often running into a OOM event

slurmstepd: error: Detected 1 oom-kill event(s) in step 7560833.batch cgroup.
j-berg commented 5 years ago

Errors and OOM events seem to be fixed with 7d351c8f3535cdaafa32974d7ea9535d99d79c0e and 55e5da6b3e2bbd758c33637e77194362c8d53279, respectively.