search

MiraldiLab / maxATAC

Transcription Factor Binding Prediction from ATAC-seq and scATAC-seq with Deep Neural Networks
Apache License 2.0
25 stars 8 forks source link

`maxatac prepare` erroring out at normalization #96

Closed codymowery closed 2 years ago

codymowery commented 2 years ago

Hello Miraldi Lab! Thanks for generating such a valuable tool, and thanks in advance for your help! I followed installation directions into a dedicated conda environment and began running maxatac prepare on some public ATAC data (i.e. SRR7650829) aligned to hg38 with bowtie2. Two things:

First, I think it'd be helpful to list specific package versions for the conda install. I had some issues with the default samtools version and ended up having to upgrade to 1.15.1 in order for the pre-normalization functions to work properly.

Second, I haven't been able to resolve this error during the normalization step. I appreciate any suggestions you can offer!

image

Package versions from conda list here:

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge absl-py 1.0.0 pypi_0 pypi astunparse 1.6.3 pypi_0 pypi bedtools 2.30.0 h468198e_3 bioconda biopython 1.79 pypi_0 pypi bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.18.1 h7f8727e_0 ca-certificates 2021.10.8 ha878542_0 conda-forge cachetools 5.0.0 pypi_0 pypi certifi 2021.10.8 pypi_0 pypi charset-normalizer 2.0.12 pypi_0 pypi curl 7.83.1 h7bff187_0 conda-forge cycler 0.11.0 pypi_0 pypi flatbuffers 2.0 pypi_0 pypi fonttools 4.33.3 pypi_0 pypi gast 0.5.3 pypi_0 pypi google-auth 2.6.6 pypi_0 pypi google-auth-oauthlib 0.4.6 pypi_0 pypi google-pasta 0.2.0 pypi_0 pypi grpcio 1.46.1 pypi_0 pypi h5py 3.6.0 pypi_0 pypi htslib 1.15.1 h9753748_0 bioconda idna 3.3 pypi_0 pypi importlib-metadata 4.11.3 pypi_0 pypi joblib 1.1.0 pypi_0 pypi keras 2.8.0 pypi_0 pypi keras-preprocessing 1.1.2 pypi_0 pypi keyutils 1.6.1 h166bdaf_0 conda-forge kiwisolver 1.4.2 pypi_0 pypi krb5 1.19.3 h3790be6_0 conda-forge ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge libclang 14.0.1 pypi_0 pypi libcurl 7.83.1 h7bff187_0 conda-forge libdeflate 1.10 h7f98852_0 conda-forge libedit 3.1.20210910 h7f8727e_0 libev 4.33 h7f8727e_1 libffi 3.4.2 h7f98852_5 conda-forge libgcc 7.2.0 h69d50b8_2 conda-forge libgcc-ng 12.1.0 h8d9b700_16 conda-forge libgomp 12.1.0 h8d9b700_16 conda-forge libidn2 2.3.2 h7f8727e_0 libnghttp2 1.47.0 h727a467_0 conda-forge libnsl 2.0.0 h7f98852_0 conda-forge libpng 1.6.37 h21135ba_2 conda-forge libssh2 1.10.0 ha56f1ee_2 conda-forge libstdcxx-ng 12.1.0 ha89aaad_16 conda-forge libunistring 0.9.10 h7f98852_0 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libzlib 1.2.11 h166bdaf_1014 conda-forge markdown 3.3.7 pypi_0 pypi matplotlib 3.5.2 pypi_0 pypi maxatac 1.0.5 pypi_0 pypi mysql-connector-c 6.1.11 h6eb9d5d_1007 conda-forge ncurses 6.3 h7f8727e_2 numpy 1.22.3 pypi_0 pypi oauthlib 3.2.0 pypi_0 pypi openssl 1.1.1o h7f8727e_0 opt-einsum 3.3.0 pypi_0 pypi packaging 21.3 pypi_0 pypi pandas 1.4.2 pypi_0 pypi pigz 2.6 h27cfd23_0 pillow 9.1.0 pypi_0 pypi pip 22.1 pyhd8ed1ab_0 conda-forge protobuf 3.20.1 pypi_0 pypi py2bit 0.3.0 pypi_0 pypi pyasn1 0.4.8 pypi_0 pypi pyasn1-modules 0.2.8 pypi_0 pypi pybedtools 0.9.0 pypi_0 pypi pybigwig 0.3.18 pypi_0 pypi pydot 1.4.2 pypi_0 pypi pyfiglet 0.8.post1 pypi_0 pypi pyparsing 3.0.9 pypi_0 pypi pysam 0.19.0 pypi_0 pypi python 3.9.12 h9a8a25e_1_cpython conda-forge python-dateutil 2.8.2 pypi_0 pypi python_abi 3.9 2_cp39 conda-forge pytz 2022.1 pypi_0 pypi pyyaml 6.0 pypi_0 pypi readline 8.1.2 h7f8727e_1 requests 2.27.1 pypi_0 pypi requests-oauthlib 1.3.1 pypi_0 pypi rsa 4.8 pypi_0 pypi samtools 1.15.1 h1170115_0 bioconda scikit-learn 1.1.0 pypi_0 pypi scipy 1.8.0 pypi_0 pypi seaborn 0.11.2 pypi_0 pypi setuptools 62.2.0 py39hf3d152e_0 conda-forge six 1.16.0 pypi_0 pypi sqlite 3.38.5 h4ff8645_0 conda-forge tensorboard 2.8.0 pypi_0 pypi tensorboard-data-server 0.6.1 pypi_0 pypi tensorboard-plugin-wit 1.8.1 pypi_0 pypi tensorflow 2.8.0 pypi_0 pypi tensorflow-io-gcs-filesystem 0.25.0 pypi_0 pypi termcolor 1.1.0 pypi_0 pypi tf-estimator-nightly 2.8.0.dev2021122109 pypi_0 pypi threadpoolctl 3.1.0 pypi_0 pypi tk 8.6.12 h27826a3_0 conda-forge typing-extensions 4.2.0 pypi_0 pypi tzdata 2022a hda174b7_0 ucsc-bedgraphtobigwig 377 ha8a8165_3 bioconda urllib3 1.26.9 pypi_0 pypi werkzeug 2.1.2 pypi_0 pypi wget 1.20.3 ha56f1ee_1 conda-forge wheel 0.37.1 pyhd8ed1ab_0 conda-forge wrapt 1.14.1 pypi_0 pypi xz 5.2.5 h7f8727e_1 zipp 3.8.0 pypi_0 pypi zlib 1.2.11 h166bdaf_1014 conda-forge

tacazares commented 2 years ago

Hey @codymowery , Thanks for posting. I have run into the pybigwig issue before: https://github.com/MiraldiLab/maxATAC/issues/87#issue-1139117054

I think it has to do with pybigwig not recognizing or finding numpy when installing.

Test if this is the issue by activating your env. Then you need to open your python interpreter and test if pybigwig can find numpy:

import pyBigWig
pybigwig.numpy

If 0 is returned then you need to try to pip uninstall pybigwig then pip install pybigwig.

I have had this issue on HPC systems so far and a few version of Mac OS.

Let me know if this works! Tareian

codymowery commented 2 years ago

Thanks @tacazares, and sorry I missed the other issue. That seems to have fixed it. I did have to specify pip install pybigwig --no-cache-dir to avoid re-installing from the problematic package.

Another suggestion -- It would be nice to update the README.md to specify the maxatac prepare output that is subsequently passed to maxatac predict. I was a little confused by maxatac predict --sequence hg38.2bit -tf CTCF --signal GM12878.bigwig -o outputdir/ in the README since maxatac prepare wasn't outputting a ${prefix}.bigwig file. Your schematic figure suggests it's the normalized ${prefix}_IS_slop20_RP20M_minmax01.bw output that goes into maxatac predict, but updating the examples would avoid potential confusion for others.

Thanks again for creating and maintaining the package! I'm super excited to see how the data look.