r3fang / SnapATAC

Analysis Pipeline for Single Cell ATAC-seq
GNU General Public License v3.0
301 stars 125 forks source link

problems with runMACS #110

Closed kchowdhary4 closed 4 years ago

kchowdhary4 commented 4 years ago

Hi, I've trying to run the runMACS command in order to call peaks on my clusters. However, I keep running into problems with there not being snaptools.parser available or problems importing the correct numpy multiarray_umath.

Here is my code :

`clusters.sel = names(table(x.sp@cluster))[which(table(x.sp@cluster) > 100)];

peaks.ls = mclapply(seq(clusters.sel), function(i){ print(clusters.sel[i]); runMACS( obj=x.sp[which(x.sp@cluster==clusters.sel[i]),], output.prefix=paste0("teffatac.", gsub(" ", "", clusters.sel)[i]), path.to.snaptools="/anaconda/envs/py3/bin/snaptools", path.to.macs="/anaconda/envs/py3/bin/macs2", gsize="mm", # mm, hs, etc buffer.size=500, num.cores=1, macs.options="--nomodel --shift 37 --ext 73 --qval 1e-2 -B --SPMR --call-summits", tmp.folder=tempdir() ); }, mc.cores=1);`

Here is the output I get:

`Epoch: checking input parameters ... Epoch: extracting fragments from each snap files ... Traceback (most recent call last): File "/anaconda/envs/py3/bin/snaptools", line 36, in from snaptools.parser import parse_args ImportError: No module named snaptools.parser cat: /var/folders/vc/k9mh38dn1_5bslrm1ztjwlz00000gn/T//RtmpUArhVg/filef105a1f7b40.bed.gz: No such file or directory Traceback (most recent call last): File "/anaconda/envs/py3/bin/macs2", line 650, in main() File "/anaconda/envs/py3/bin/macs2", line 48, in main from MACS2.callpeak_cmd import run File "/anaconda/envs/py3/lib/python2.7/site-packages/MACS2/callpeak_cmd.py", line 23, in from MACS2.OptValidator import opt_validate File "/anaconda/envs/py3/lib/python2.7/site-packages/MACS2/OptValidator.py", line 20, in from MACS2.IO.Parser import BEDParser, ELANDResultParser, ELANDMultiParser, \ File "init.pxd", line 206, in init MACS2.IO.Parser File "/anaconda/envs/py3/lib/python2.7/site-packages/numpy/init.py", line 142, in from . import core File "/anaconda/envs/py3/lib/python2.7/site-packages/numpy/core/init.py", line 71, in raise ImportError(msg) ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the multiarray numpy extension module failed. Most likely you are trying to import a failed build of numpy. Here is how to proceed:

Original error was: dlopen(/anaconda/envs/py3/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so, 2): Library not loaded: @rpath/libopenblas.dylib Referenced from: /anaconda/envs/py3/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so Reason: image not found

Error in runMACS(obj = x.sp[which(x.sp@cluster == clusters.sel[i]), ], : 'MACS' call failed`

I've tried everything from changing my python environments, reinstalling snaptools/numpy/macs2, etc.

Any help on this would be very much appreciated -- thank you!

r3fang commented 4 years ago

can you test if MACS has been successfully installed by running this command “./dir/macs2”, what is the outcome of this command?

-- Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Oct 1, 2019, at 5:15 PM, kchowdhary4 notifications@github.com wrote:

Hi, I've trying to run the runMACS command in order to call peaks on my clusters. However, I keep running into problems with there not being snaptools.parser available or problems importing the correct numpy multiarray_umath.

Here is my code :

clusters.sel = names(table(x.sp@cluster))[which(table(x.sp@cluster) > 100)]; peaks.ls = mclapply(seq(clusters.sel), function(i){ print(clusters.sel[i]); runMACS( obj=x.sp[which(x.sp@cluster==clusters.sel[i]),], output.prefix=paste0("teffatac.", gsub(" ", "", clusters.sel)[i]), path.to.snaptools="/anaconda/envs/py3/bin/snaptools", path.to.macs="/anaconda/envs/py3/bin/macs2", gsize="mm", # mm, hs, etc buffer.size=500, num.cores=1, macs.options="--nomodel --shift 37 --ext 73 --qval 1e-2 -B --SPMR --call-summits", tmp.folder=tempdir() ); }, mc.cores=1);

Here is the output I get:

`Epoch: checking input parameters ... Epoch: extracting fragments from each snap files ... Traceback (most recent call last): File "/anaconda/envs/py3/bin/snaptools", line 36, in from snaptools.parser import parse_args ImportError: No module named snaptools.parser cat: /var/folders/vc/k9mh38dn1_5bslrm1ztjwlz00000gn/T//RtmpUArhVg/filef105a1f7b40.bed.gz: No such file or directory Traceback (most recent call last): File "/anaconda/envs/py3/bin/macs2", line 650, in main() File "/anaconda/envs/py3/bin/macs2", line 48, in main from MACS2.callpeak_cmd import run File "/anaconda/envs/py3/lib/python2.7/site-packages/MACS2/callpeak_cmd.py", line 23, in from MACS2.OptValidator import opt_validate File "/anaconda/envs/py3/lib/python2.7/site-packages/MACS2/OptValidator.py", line 20, in from MACS2.IO.Parser import BEDParser, ELANDResultParser, ELANDMultiParser, File "init.pxd", line 206, in init MACS2.IO.Parser File "/anaconda/envs/py3/lib/python2.7/site-packages/numpy/init.py", line 142, in from . import core File "/anaconda/envs/py3/lib/python2.7/site-packages/numpy/core/init.py", line 71, in raise ImportError(msg) ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the multiarray numpy extension module failed. Most likely you are trying to import a failed build of numpy. Here is how to proceed:

If you're working with a numpy git repository, try git clean -xdf (removes all files not under version control) and rebuild numpy. If you are simply trying to use the numpy version that you have installed: your installation is broken - please reinstall numpy. If you have already reinstalled and that did not fix the problem, then: Check that you are using the Python you expect (you're using /anaconda/envs/py3/bin/python), and that you have no directories in your PATH or PYTHONPATH that can interfere with the Python and numpy versions you're trying to use.

If (1) looks fine, you can open a new issue at https://github.com/numpy/numpy/issues https://github.com/numpy/numpy/issues. Please include details on:

how you installed Python how you installed numpy your operating system whether or not you have multiple versions of Python installed if you built from source, your compiler versions and ideally a build log Note: this error has many possible causes, so please don't comment on an existing issue about this - open a new one instead.

Original error was: dlopen(/anaconda/envs/py3/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so, 2): Library not loaded: @rpath/libopenblas.dylib Referenced from: /anaconda/envs/py3/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so Reason: image not found

Error in runMACS(obj = x.sp[which(x.sp@cluster == clusters.sel[i]), ], : 'MACS' call failed`

I've tried everything from changing my python environments, reinstalling snaptools/numpy/macs2, etc.

Any help on this would be very much appreciated -- thank you!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/110?email_source=notifications&email_token=ABT6GG5W3CQ4DH4GTIRMKODQMO4VXA5CNFSM4I4O5WQ2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HO66DVQ, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GG5SZZJEKB4LKUSJERLQMO4VXANCNFSM4I4O5WQQ.

kchowdhary4 commented 4 years ago

Thanks for the quick reply!

When I run “./dir/macs2”, I get "No such file or directory"

However, when I try to install MACS2 via pip or conda, I get: "Requirement already satisfied: MACS2 in /anaconda/envs/py3/lib/python2.7/site-packages (2.1.4)"

r3fang commented 4 years ago

sorry for not being very clear. Can you replace the “dir/“ to specific folder that contains the executable MACS2 file? I just want to make sure that MACS2 is installed and executable.

Best,

Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Oct 1, 2019, at 5:27 PM, kchowdhary4 notifications@github.com wrote:

Thanks for the quick reply!

When I run “./dir/macs2”, I get "No such file or directory"

However, when I try to install MACS2 via pip or conda, I get: "Requirement already satisfied: MACS2 in /anaconda/envs/py3/lib/python2.7/site-packages (2.1.4)"

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/110?email_source=notifications&email_token=ABT6GG3TBG6G7KYAD4RUN3LQMO6CPA5CNFSM4I4O5WQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEACZS5A#issuecomment-537237876, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GGYCPA7Z6FJL5JPD3RLQMO6CPANCNFSM4I4O5WQQ.

kchowdhary4 commented 4 years ago

Got it --

When i change directory to the one containing the executable (/anaconda/envs/py3/bin) and then run "./macs2", I get

"usage: macs2 [-h] [--version] {callpeak,bdgpeakcall,bdgbroadcall,bdgcmp,bdgopt,cmbreps,bdgdiff,filterdup,predictd,pileup,randsample,refinepeak} ... macs2: error: too few arguments"

r3fang commented 4 years ago

what if you run "snaptools"? it seems that "snaptools" failed

kchowdhary4 commented 4 years ago

if I run "snaptools" or "./snaptools" I get: "Traceback (most recent call last): File "./snaptools", line 36, in from snaptools.parser import parse_args ImportError: No module named snaptools.parser "

r3fang commented 4 years ago

what's your python version?

kchowdhary4 commented 4 years ago

2.7.16

kchowdhary4 commented 4 years ago

Just tried creating a completely new python 2.7 environment and reinstalling pysam, h5py, numpy, pybedtools, macs2 in this environment. Then installed snaptools via git ($ git clone https://github.com/r3fang/snaptools.git $ cd snaptools $ pip install -e .)

Snaptools installed but then when I do ./bin/snaptools, I get

" Traceback (most recent call last): File "./bin/snaptools", line 36, in from snaptools.parser import parse_args File "/Users/Documents/data_analysis/snaptools/snaptools/parser.py", line 31, in from snaptools.utilities import str2bool File "/Users/Documents/data_analysis/snaptools/snaptools/utilities.py", line 5, in import pysam File "/anaconda/lib/python2.7/site-packages/pysam/init.py", line 5, in from pysam.libchtslib import * ImportError: dlopen(/anaconda/lib/python2.7/site-packages/pysam/libchtslib.so, 2): Library not loaded: @rpath/libdeflate.so Referenced from: /anaconda/lib/libhts.2.dylib Reason: image not found"

r3fang commented 4 years ago

can you install snap tools using “pip install snaptools”?

Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Oct 1, 2019, at 6:03 PM, kchowdhary4 notifications@github.com wrote:

Just tried creating a completely new python 2.7 environment and reinstalling pysam, h5py, numpy, pybedtools in this environment. Then installed snaptools via git ($ git clone https://github.com/r3fang/snaptools.git https://github.com/r3fang/snaptools.git $ cd snaptools $ pip install -e .)

Snaptools installed but then when I do ./snaptools, I get

" Traceback (most recent call last): File "./bin/snaptools", line 36, in from snaptools.parser import parse_args File "/Users/Documents/data_analysis/snaptools/snaptools/parser.py", line 31, in from snaptools.utilities import str2bool File "/Users/Documents/data_analysis/snaptools/snaptools/utilities.py", line 5, in import pysam File "/anaconda/lib/python2.7/site-packages/pysam/init.py", line 5, in from pysam.libchtslib import * ImportError: dlopen(/anaconda/lib/python2.7/site-packages/pysam/libchtslib.so, 2): Library not loaded: @rpath/libdeflate.so Referenced from: /anaconda/lib/libhts.2.dylib Reason: image not found"

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/110?email_source=notifications&email_token=ABT6GG3IYEFGYSFBY3HOJBDQMPCJ7A5CNFSM4I4O5WQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAC5CXI#issuecomment-537252189, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GG75FS5A5NI42TOPF3TQMPCJ7ANCNFSM4I4O5WQQ.

kchowdhary4 commented 4 years ago

Yes, I tried that as well -- leads to the following error:

"Traceback (most recent call last): File "/anaconda/envs/py27/bin/snaptools", line 36, in from snaptools.parser import parse_args File "/anaconda/envs/py27/lib/python2.7/site-packages/snaptools/parser.py", line 31, in from snaptools.utilities import str2bool File "/anaconda/envs/py27/lib/python2.7/site-packages/snaptools/utilities.py", line 5, in import pysam File "/anaconda/envs/py27/lib/python2.7/site-packages/pysam/init.py", line 5, in from pysam.libchtslib import * ImportError: dlopen(/anaconda/envs/py27/lib/python2.7/site-packages/pysam/libchtslib.so, 2): Library not loaded: @rpath/libdeflate.so Referenced from: /anaconda/envs/py27/lib/libhts.2.dylib Reason: image not found"

r3fang commented 4 years ago

but the installation is fine?

-- Rongxin Fang Ph.D. Student, Ren Lab Ludwig Institute for Cancer Research University of California, San Diego

On Oct 2, 2019, at 9:21 AM, kchowdhary4 notifications@github.com wrote:

Yes, I tried that as well -- leads to the following error:

"Traceback (most recent call last): File "/anaconda/envs/py27/bin/snaptools", line 36, in from snaptools.parser import parse_args File "/anaconda/envs/py27/lib/python2.7/site-packages/snaptools/parser.py", line 31, in from snaptools.utilities import str2bool File "/anaconda/envs/py27/lib/python2.7/site-packages/snaptools/utilities.py", line 5, in import pysam File "/anaconda/envs/py27/lib/python2.7/site-packages/pysam/init.py", line 5, in from pysam.libchtslib import * ImportError: dlopen(/anaconda/envs/py27/lib/python2.7/site-packages/pysam/libchtslib.so, 2): Library not loaded: @rpath/libdeflate.so Referenced from: /anaconda/envs/py27/lib/libhts.2.dylib Reason: image not found"

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/110?email_source=notifications&email_token=ABT6GG4HFYDGZX42U2JZ3X3QMSN5NA5CNFSM4I4O5WQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAEW5FA#issuecomment-537489044, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GG4YKVOHP6H3UGSW6RLQMSN5NANCNFSM4I4O5WQQ.

kchowdhary4 commented 4 years ago

yup -- no problems with the installation. I get the above error when I do "snaptools"

r3fang commented 4 years ago

umm. if snaptools does not work then how did you generate the snap file?

On Oct 2, 2019, at 9:38 AM, kchowdhary4 notifications@github.com wrote:

yup -- no problems with the installation. I get the above error when I do "snaptools"

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/r3fang/SnapATAC/issues/110?email_source=notifications&email_token=ABT6GG7MH3PBTM5Q2JFWDQDQMSP47A5CNFSM4I4O5WQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAEYTAQ#issuecomment-537495938, or mute the thread https://github.com/notifications/unsubscribe-auth/ABT6GG4WIMEHH5GSXSNURCLQMSP47ANCNFSM4I4O5WQQ.

kchowdhary4 commented 4 years ago

actually, I ended up figuring it out -- it was a problem with condasolver -- things seem to be working now. Thanks for your help!