runmacsforall error python locale encoding

KyleFerchen commented 5 years ago

Hello,

I generated a snap file from cellranger bam output. Then I used snapatac to cluster the cells. However, now when I try to run the runMACSForAll() function, I get the following errors. Is there something else I need to install for the python code to run properly?

Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: checking input parameters ...
Epoch: extracting fragments from each snap files ...
Epoch: extracting fragments from each snap files ...
Fatal Python error: Py_Initialize: Unable to get the locale encoding
File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002b89d875a940 (most recent call first): Epoch: extracting fragments from each snap files ...
Epoch: extracting fragments from each snap files ...
Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002b357bcc3940 (most recent call first): Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002ae4b2dd3940 (most recent call first): Epoch: extracting fragments from each snap files ...
Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002ac2cea7e940 (most recent call first): cat: /tmp/RtmpiYdj3N/file9c57222969d.bed.gz: No such file or directory Epoch: extracting fragments from each snap files ...
Epoch: extracting fragments from each snap files ...
Epoch: extracting fragments from each snap files ...
cat: /tmp/RtmpiYdj3N/fileb597222969d.bed.gz: No such file or directory cat: /tmp/RtmpiYdj3N/filea067222969d.bed.gz: No such file or directory Fatal Python error: Py_Initialize: Unable to get the locale encoding
File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002b90d2200940 (most recent call first): Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002aab0111c940 (most recent call first): cat: /tmp/RtmpiYdj3N/fileac67222969d.bed.gz: No such file or directory Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in
import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
Epoch: extracting fragments from each snap files ...
Fatal Python error: Py_Initialize: Unable to get the locale encoding
File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002b3eea938940 (most recent call first): Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002b69b0418940 (most recent call first): Epoch: extracting fragments from each snap files ...
Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
cat: /tmp/RtmpiYdj3N/fileb6f7222969d.bed.gz: No such file or directory
Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in
import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
cat: /tmp/RtmpiYdj3N/filea447222969d.bed.gz: No such file or directory
Fatal Python error: Py_Initialize: Unable to get the locale encoding
File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002b77a3c3a940 (most recent call first): Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
cat: /tmp/RtmpiYdj3N/filea687222969d.bed.gz: No such file or directory
Fatal Python error: Py_Initialize: Unable to get the locale encoding
File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002af0b68a0940 (most recent call first): cat: /tmp/RtmpiYdj3N/filebb77222969d.bed.gz: No such file or directory Epoch: extracting fragments from each snap files ...
Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in
import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
cat: /tmp/RtmpiYdj3N/filec377222969d.bed.gz: No such file or directory
Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in
import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in
import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
cat: /tmp/RtmpiYdj3N/filec5c7222969d.bed.gz: No such file or directory
Epoch: extracting fragments from each snap files ...
Fatal Python error: Py_Initialize: Unable to get the locale encoding
File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002b3709e39940 (most recent call first): Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
Epoch: extracting fragments from each snap files ...
Epoch: extracting fragments from each snap files ...
Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in
import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
cat: /tmp/RtmpiYdj3N/filec467222969d.bed.gz: No such file or directory
Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in
import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
Fatal Python error: Py_Initialize: Unable to get the locale encoding
File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002ba948910940 (most recent call first): Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002ac727b77940 (most recent call first): Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Current thread 0x00002b5ff2356940 (most recent call first): Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in import argparse as ap
File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in
import collections as _collections
File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in
from _collections import deque, defaultdict
ImportError: No module named _collections
cat: /tmp/RtmpiYdj3N/filec517222969d.bed.gz: No such file or directory
cat: /tmp/RtmpiYdj3N/filec3e7222969d.bed.gz: No such file or directory
cat: /tmp/RtmpiYdj3N/filec0d7222969d.bed.gz: No such file or directory
Traceback (most recent call last):
File "/usr/local/python/2.7.14/bin/macs2", line 25, in import argparse as ap File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in import collections as _collections File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in from _collections import deque, defaultdict ImportError: No module named _collections Traceback (most recent call last): File "/usr/local/python/2.7.14/bin/macs2", line 25, in import argparse as ap File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in import collections as _collections File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in from _collections import deque, defaultdict ImportError: No module named _collections Traceback (most recent call last): File "/usr/local/python/2.7.14/bin/macs2", line 25, in import argparse as ap File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in import collections as _collections File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in from _collections import deque, defaultdict ImportError: No module named _collections Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘reduce’ for signature ‘"list"’ In addition: Warning message: In parallel::mclapply(as.list(levels(obj@cluster)), function(x) { : scheduled core 1, 2, 7, 5, 8, 3, 4, 9, 11, 15, 13, 14, 10, 12 encountered error in user code, all values of the job will be affected

r3fang commented 5 years ago

Dear Kyle,

Can you run runMACS for one cluster? Does it work?

Best, -R

KyleFerchen commented 5 years ago

No, it still runs into the same python error:

Epoch: checking input parameters ... Epoch: extracting fragments from each snap files ... Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/usr/local/python/2.7.14/lib/python2.7/encodings/init.py", line 123 raise CodecRegistryError,\ ^ SyntaxError: invalid syntax

Current thread 0x00002b233eec7940 (most recent call first): cat: /tmp/RtmpMQmRgs/file645f2cae31ef.bed.gz: No such file or directory Traceback (most recent call last): File "/usr/local/python/2.7.14/bin/macs2", line 25, in import argparse as ap File "/usr/local/python/2.7.14/lib/python2.7/argparse.py", line 85, in import collections as _collections File "/usr/local/python/2.7.14/lib/python2.7/collections.py", line 20, in from _collections import deque, defaultdict ImportError: No module named _collections Error in runMACS(obj = x.sp[which(x.sp@cluster == 9), ], output.prefix = "snap_grimes_3531", : 'MACS' call failed

r3fang commented 5 years ago

you said you were able to create snap file using snaptools pre?

r3fang commented 5 years ago

it seems something wrong with your python environment path

https://askubuntu.com/questions/640010/fatal-python-error-py-initialize-unable-to-get-the-locale-encoding-syntaxe https://github.com/GNS3/gns3-gui/issues/2267

Both point to the same kinda error

-R

r3fang commented 5 years ago

please let me know if this does not solve the problem or anything else that i can help

KyleFerchen commented 5 years ago

I'm trying to work through this issue. I'm not exactly sure what I need to set the PYTHONPATH variable to. In order to use the SNAP-ATAC on our scATAC-seq data, I have to use a Centos computing cluster to get enough memory. I'm loading the modules for MACS2 which is in python2 and snap-atac in python3. Maybe this is where the problem is coming from.

I have no idea what to set the PYTHONPATH to though.

KyleFerchen commented 5 years ago

I worked around the issue by calling snaptools dump-fragment to isolate the fragments associated with each cluster based on the snap-object. Then I ran MACS2 separately to generate peak files for each cluster. I was then able to create the pmat for the snap object.

Ex.

    snaptools dump-fragment --snap-file diff_gene_pair_3531.snap \
           --output-file snap_atac_diff_3531_cluster_bam/diff_gene_pair_3531_cluster_1.bam \
           --barcode-file snap_atac_diff_3531_barcode_files/cluster_1.txt

    cd snap_atac_diff_3531_cluster_bam/

    macs2 callpeak -t diff_gene_pair_3531_cluster_1.bed -f BED -g mm \
            -n diff_gene_snap_cluster_1 --nomodel --shift 37 --ext 73 \
            --qval 1e-2 -B --SPMR --call-summits

... continue running snap-atac analysis in R

However, is there an option to find differential accessible regions for each cluster by comparing the cluster to all of the remaining cells? I'm worried I miss important dynamic changes the way the cells are selected with the NULL option for remaining cells when calculating DARs. How do I know what my cells of interest are being compared to?

r3fang / SnapATAC

runmacsforall error python locale encoding #22