search

kundajelab / chrombpnet

Bias factorized, base-resolution deep learning models of chromatin accessibility (chromBPNet)
https://github.com/kundajelab/chrombpnet/wiki
MIT License
112 stars 30 forks source link

Error in importance_hdf5_to_bigwig.py requiring region_used argument #150

Closed tli71193 closed 9 months ago

tli71193 commented 10 months ago

Hi there

Ran into an error when running the contrib function where in importance_hdf5_to_bigwig.py

Traceback (most recent call last):
  File "/net/gs/vol1/home/tli824/bin/software/chrombpnet/chrombpnet/evaluation/make_bigwigs/importance_hdf5_to_bigwig.py", line 43, in <module>
    main(args)
  File "/net/gs/vol1/home/tli824/bin/software/chrombpnet/chrombpnet/evaluation/make_bigwigs/importance_hdf5_to_bigwig.py", line 30, in main
    regions = bigwig_helper.get_regions(args.regions, SEQLEN)
TypeError: get_regions() missing 1 required positional argument: 'regions_used'

I created a temporary work around/patch in the meantime by editing the get_regions function where regions_used = None by default

def get_regions(regions_file, seqlen, regions_used = None):
    # regions file is assumed to be centered at summit (2nd + 10th column)
    # it is adjusted to be of length seqlen centered at summit

    assert(seqlen%2==0)

    with open(regions_file) as r:
        regions = [x.strip().split('\t') for x in r]
    if regions_used:
        regions = [[x[0], int(x[1])+int(x[9])-seqlen//2, int(x[1])+int(x[9])+seqlen//2, int(x[1])+int(x[9])] for x in np.array(regions)[regions_used]]
    else:
        regions = [[x[0], int(x[1])+int(x[9])-seqlen//2, int(x[1])+int(x[9])+seqlen//2, int(x[1])+int(x[9])] for x in np.array(regions)]

    return regions
snaqvi1990 commented 10 months ago

I have this same error

panushri25 commented 10 months ago

Hello,

Thank you for this, I will fix it today and keep you all posted.

panushri25 commented 10 months ago

Hello all,

Fixed this and made a new release. Feel free to reply back here or open another issue if you continue to see any errors or have questions.

Thank you, Anu

robinmeyers commented 10 months ago

Hi @panushri25 ,

I think this fix may have caused a new error when running chrombpnet contribs_bw since nothing is getting passed as regions_used.

Traceback (most recent call last):
  File "/opt/conda/bin/chrombpnet", line 33, in <module>
    sys.exit(load_entry_point('chrombpnet', 'console_scripts', 'chrombpnet')())
  File "/scratch/chrombpnet/chrombpnet/CHROMBPNET.py", line 68, in main
    importance_hdf5_to_bigwig.main(args_copy)
  File "/scratch/chrombpnet/chrombpnet/evaluation/make_bigwigs/importance_hdf5_to_bigwig.py", line 30, in main
    regions = bigwig_helper.get_regions(args.regions, SEQLEN)
  File "/scratch/chrombpnet/chrombpnet/evaluation/make_bigwigs/bigwig_helper.py", line 41, in get_regions
    regions = [[x[0], int(x[1])+int(x[9])-seqlen//2, int(x[1])+int(x[9])+seqlen//2, int(x[1])+int(x[9])] for x in np.array(regions.values)[regions_used]]
  File "/scratch/chrombpnet/chrombpnet/evaluation/make_bigwigs/bigwig_helper.py", line 41, in <listcomp>
    regions = [[x[0], int(x[1])+int(x[9])-seqlen//2, int(x[1])+int(x[9])+seqlen//2, int(x[1])+int(x[9])] for x in np.array(regions.values)[regions_used]]
TypeError: only size-1 arrays can be converted to Python scalars

Best, Robin

panushri25 commented 10 months ago

thank you for this, let me take a look. I did it test it even with empty regions_used, maybe its version specific.

Can you tell me your numpy version?

robinmeyers commented 10 months ago

Numpy version is 1.23.4

Rest of my package versions if it's helpful:

absl-py                      2.0.0
asttokens                    2.4.1
astunparse                   1.6.3
blosc2                       2.3.1
boltons                      23.0.0
brotlipy                     0.7.0
cachetools                   5.3.2
cairocffi                    1.6.1
CairoSVG                     2.7.1
certifi                      2023.7.22
cffi                         1.15.0
charset-normalizer           2.0.4
chrombpnet                   0.1.5               /scratch/chrombpnet
cloudpickle                  3.0.0
colorama                     0.4.4
conda                        23.7.4
conda-content-trust          0+unknown
conda-package-handling       1.8.1
contourpy                    1.2.0
cryptography                 36.0.0
cssselect2                   0.7.0
cycler                       0.12.1
decorator                    5.1.1
deepdish                     0.3.7
deeplift                     0.6.13.0
defusedxml                   0.7.1
dm-tree                      0.1.8
exceptiongroup               1.1.3
executing                    2.0.1
flatbuffers                  23.5.26
fonttools                    4.44.0
gast                         0.5.4
google-auth                  2.23.4
google-auth-oauthlib         0.4.6
google-pasta                 0.2.0
grpcio                       1.59.2
h5py                         3.10.0
hdf5plugin                   4.3.0
html5lib                     1.1
idna                         3.3
igraph                       0.9.11
imageio                      2.32.0
importlib-metadata           6.8.0
importlib-resources          6.1.1
ipython                      8.17.2
jedi                         0.19.1
joblib                       1.3.2
jsonpatch                    1.33
jsonpointer                  2.4
keras                        2.8.0
Keras-Preprocessing          1.1.2
kiwisolver                   1.4.5
kundajelab-shap              1
lazy_loader                  0.3
leidenalg                    0.8.10
libclang                     16.0.6
llvmlite                     0.41.1
logomaker                    0.8
Markdown                     3.5.1
MarkupSafe                   2.1.3
matplotlib                   3.8.1
matplotlib-inline            0.1.6
matplotlib-venn              0.11.6
modisco                      0.5.16.0
modisco-lite                 2.0.7
msgpack                      1.0.7
ndindex                      1.7
networkx                     3.2.1
numba                        0.58.1
numexpr                      2.8.7
numpy                        1.23.4
oauthlib                     3.2.2
opt-einsum                   3.3.0
packaging                    23.2
pandas                       2.1.3
parso                        0.8.3
pexpect                      4.8.0
Pillow                       10.0.1
pip                          23.3.1
pluggy                       1.3.0
prompt-toolkit               3.0.40
protobuf                     3.20.0
psutil                       5.9.6
ptyprocess                   0.7.0
pure-eval                    0.2.2
py-cpuinfo                   9.0.0
pyasn1                       0.5.0
pyasn1-modules               0.3.0
pyBigWig                     0.3.18
pycosat                      0.6.3
pycparser                    2.21
pyfaidx                      0.6.1
Pygments                     2.16.1
pyOpenSSL                    22.0.0
pyparsing                    3.1.1
pyphen                       0.14.0
PySocks                      1.7.1
python-dateutil              2.8.2
pytz                         2023.3.post1
requests                     2.27.1
requests-oauthlib            1.3.1
rsa                          4.9
ruamel.yaml                  0.17.10
ruamel.yaml.clib             0.2.2
ruamel-yaml-conda            0.15.100
scikit-image                 0.22.0
scikit-learn                 1.3.2
scipy                        1.11.3
setuptools                   61.2.0
six                          1.16.0
stack-data                   0.6.3
tables                       3.9.1
tensorboard                  2.8.0
tensorboard-data-server      0.6.1
tensorboard-plugin-wit       1.8.1
tensorflow                   2.8.0
tensorflow-estimator         2.8.0
tensorflow-io-gcs-filesystem 0.34.0
tensorflow-probability       0.15.0
termcolor                    2.3.0
texttable                    1.7.0
tf-estimator-nightly         2.8.0.dev2021122109
threadpoolctl                3.2.0
tifffile                     2023.9.26
tinycss2                     1.2.1
toolz                        0.12.0
tqdm                         4.48.2
traitlets                    5.13.0
typing_extensions            4.8.0
tzdata                       2023.3
urllib3                      1.26.8
wcwidth                      0.2.9
WeasyPrint                   52.5
webencodings                 0.5.1
Werkzeug                     3.0.1
wheel                        0.37.1
wrapt                        1.16.0
zipp                         3.17.0
robinmeyers commented 10 months ago

I believe the issue is that np.array()[None] adds a new axis. Using code similar to @tli71193 solved this for me.

panushri25 commented 9 months ago

@robinmeyers and @tli71193 Please update to the latest 0.1.6 version and regenerate your contribution scores. It should fix this error.

@tli71193 Your fix was helpful but it does not fully integrate the added functionality. So please try 0.1.6 and let me know if everything works okay.

robinmeyers commented 9 months ago

@panushri25 Thanks for this fix! The latest version fixed contribs_bw for me, but now pred_bw is broken. I get this error:

Traceback (most recent call last):
  File "/opt/conda/bin/chrombpnet", line 33, in <module>
    sys.exit(load_entry_point('chrombpnet', 'console_scripts', 'chrombpnet')())
  File "/scratch/chrombpnet/chrombpnet/CHROMBPNET.py", line 56, in main
    predict_to_bigwig.main(args)
  File "/scratch/chrombpnet/chrombpnet/evaluation/make_bigwigs/predict_to_bigwig.py", line 139, in main
    regions = bigwig_helper.get_regions(args.regions, outputlen, regions_used) # output regions
  File "/scratch/chrombpnet/chrombpnet/evaluation/make_bigwigs/bigwig_helper.py", line 41, in get_regions
    if regions_used:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
panushri25 commented 9 months ago

makes sense, fixing this

panushri25 commented 9 months ago

try 0.1.7 both should be working now

YichaoOU commented 9 months ago

where is 0.1.7?

Thanks, Yichao

panushri25 commented 9 months ago

Have you been installing using pip or from github?

YichaoOU commented 9 months ago

Thanks for the quick reply! In the GitHub Release, there is no "0.1.7".

Yes. I do find it in pip. https://pypi.org/project/chrombpnet/

Thanks, Yichao

panushri25 commented 9 months ago

yes, i will make it the latest on github once i close this issue

robinmeyers commented 9 months ago

0.1.7 fixed the issue for me. Thanks!

panushri25 commented 9 months ago

Awesome thank for the update!