search

daler / metaseq

Framework for integrated analysis and plotting of ChIP/RIP/RNA/*-seq data
https://daler.github.io/metaseq
MIT License
87 stars 36 forks source link

multiprocessing #28

Open kellermac opened 8 years ago

kellermac commented 8 years ago

Hi again! I was hoping you might be able to trace the root issue of my problem. I am trying to plot a heatmap using a gff file for signal (this file is large 9.1Mb). I tried running it on my local machine but kept getting errors like this: NotifierThreadProc: could not create trigger pipe NotifierThreadProc: could not create trigger pipe NotifierThreadProc: could not create trigger pipe NotifierThreadProc: could not create trigger pipe NotifierThreadProc: could not create trigger pipe Traceback (most recent call last): File "heatmo.py", line 18, in Bkornabf_signal = Bkornabf_signal.array(sites_1kb, bins=100, processes=3) File "/usr/local/lib/python2.7/dist-packages/metaseq/_genomic_signal.py", line 122, in array chunksize=chunksize, **kwargs) File "/usr/local/lib/python2.7/dist-packages/metaseq/array_helpers.py", line 383, in _array_parallel itertools.repeat(kwargs))) File "/usr/lib/python2.7/multiprocessing/pool.py", line 251, in map return self.map_async(func, iterable, chunksize).get() File "/usr/lib/python2.7/multiprocessing/pool.py", line 558, in get raise self._value KeyError: "unknown preset 'empty', valid presets are 'vcf,sam,bed,psltbl,pileup,gff'"

After inspecting the genomic_signal script and converting my gff to bed I kept getting similar errors. So I moved my data to a bigger machine with 16 processors. This seemed to accommodate the "triggered pipe" but I get the following message:

/home/keller/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment. warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.') [E::bgzf_flush] hwrite error (wrong size) Traceback (most recent call last): File "heatmo.py", line 20, in Bkornabf_signal = Bkornabf_signal.array(sites_1kb, bins=100, processes=processes) File "/home/keller/anaconda2/lib/python2.7/site-packages/metaseq/_genomic_signal.py", line 122, in array chunksize=chunksize, **kwargs) File "/home/keller/anaconda2/lib/python2.7/site-packages/metaseq/array_helpers.py", line 383, in _array_parallel itertools.repeat(kwargs))) File "/home/keller/anaconda2/lib/python2.7/multiprocessing/pool.py", line 251, in map return self.map_async(func, iterable, chunksize).get() File "/home/keller/anaconda2/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value OSError: writing failed

daler commented 8 years ago

No idea, I haven't seen this before. Can you provide an example with data that reproduces the problem?

kopardev commented 5 years ago

I am getting the same error: """ Traceback (most recent call last): File "../scripts/bam2tssscore.v2.py", line 70, in bam_array=bam.array(features = tss_ext,bins = bins,processes = args.ncpu) File "/usr/local/Anaconda/envs/py2.7/lib/python2.7/site-packages/metaseq/_genomic_signal.py", line 122, in array chunksize=chunksize, **kwargs) File "/usr/local/Anaconda/envs/py2.7/lib/python2.7/site-packages/metaseq/array_helpers.py", line 383, in _array_parallel itertools.repeat(kwargs))) File "/usr/local/Anaconda/envs/py2.7/lib/python2.7/multiprocessing/pool.py", line 253, in map return self.map_async(func, iterable, chunksize).get() File "/usr/local/Anaconda/envs/py2.7/lib/python2.7/multiprocessing/pool.py", line 572, in get raise self._value KeyError: "unknown preset 'empty', valid presets are 'psltbl,gff,sam,bed,vcf'" """ Any idea? Any solutions?

kopardev commented 4 years ago

I am still getting the same error... Any solution guys?