Package dependencies don't overlap & trouble with google collab script

[bomansonja]

• MMSplice version: 2.2.0
• Python version: Python 3.8.8
• Operating System: Windows Subsystem for Linux/Google Collab

Description

Describe what you were trying to get done.

Setting up the app with

pip install cyvcf2 cython
pip install mmsplice

Tell us what happened, what went wrong, and what you expected to happen:

The following error messages regarding package dependancies:

ERROR: mmsplice 2.2.0 has requirement numpy==1.18.5, but you'll have numpy 1.21.3 which is incompatible.
ERROR: tensorflow 2.6.0 has requirement numpy~=1.19.2, but you'll have numpy 1.18.5 which is incompatible.

ERROR: kipoi 0.6.35 has requirement h5py==2.10.0, but you'll have h5py 3.1.0 which is incompatible.
ERROR: tensorflow 2.6.0 has requirement h5py~=3.1.0, but you'll have h5py 2.10.0 which is incompatible.

So the packages are dependent on different versions of other packages, hence the setup will not go through.

What I Did

To my understanding, the developers have to go under the hood to fix this issue. So I tried to run MMSplice in the google collab link, and had the following issue:

Describe what you were trying to get done.

Runt he MMSplice google collab script using the provided example vcf

Tell us what happened, what went wrong, and what you expected to happen:

When running line: predict_save(model, dl, output_csv, pathogenicity=True, splicing_efficiency=True) I get the error (traceback at bottom): ValueError: Input 0 of layer conv is incompatible with the layer: : expected min_ndim=3, found ndim=2. Full shape received: (32, 1)

What I did

Regarding the error message, I believe these could be solutions to the problem, but again ones that the authors can change: https://stackoverflow.com/questions/57430717/input-0-of-layer-conv1d-1-is-incompatible-with-the-layer-expected-ndim-3-found https://stackoverflow.com/questions/66718335/input-0-of-layer-conv1d-is-incompatible-with-the-layer-expected-min-ndim-3-f

Paste the command(s) you ran and the output (Google collab script)

predict_save(model, dl, output_csv, pathogenicity=True, splicing_efficiency=True)

If there was a crash, please include the traceback here (Google collab script)

0it [00:00, ?it/s]WARNING:tensorflow:Model was constructed with shape (None, None, 4) for input KerasTensor(type_spec=TensorSpec(shape=(None, None, 4), dtype=tf.float32, name='input_5'), name='input_5', description="created by layer 'input_5'"), but it was called on an input with incompatible shape (32, 1).
0it [00:00, ?it/s]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-13-81048a45f553> in <module>()
----> 1 predict_save(model, dl, output_csv, pathogenicity=True, splicing_efficiency=True)

14 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    992           except Exception as e:  # pylint:disable=broad-except
    993             if hasattr(e, "ag_error_metadata"):
--> 994               raise e.ag_error_metadata.to_exception(e)
    995             else:
    996               raise

ValueError: in user code:

    /usr/local/lib/python3.7/dist-packages/keras/engine/training.py:1586 predict_function  *
        return step_function(self, iterator)
    /usr/local/lib/python3.7/dist-packages/keras/engine/training.py:1576 step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py:1286 run
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py:2849 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/distribute/distribute_lib.py:3632 _call_for_each_replica
        return fn(*args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/keras/engine/training.py:1569 run_step  **
        outputs = model.predict_step(data)
    /usr/local/lib/python3.7/dist-packages/keras/engine/training.py:1537 predict_step
        return self(x, training=False)
    /usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py:1037 __call__
        outputs = call_fn(inputs, *args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/keras/engine/functional.py:415 call
        inputs, training=training, mask=mask)
    /usr/local/lib/python3.7/dist-packages/keras/engine/functional.py:550 _run_internal_graph
        outputs = node.layer(*args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py:1020 __call__
        input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
    /usr/local/lib/python3.7/dist-packages/keras/engine/input_spec.py:234 assert_input_compatibility
        str(tuple(shape)))

    ValueError: Input 0 of layer conv is incompatible with the layer: : expected min_ndim=3, found ndim=2. Full shape received: (32, 1)

PSA For future reference these are the packages that have to be installed manually when running MMSplice 2.2.0 on the Google Collab script:

!pip install --upgrade tensorflow
!pip install --upgrade numpy
from tensorflow import keras 
from tensorflow.keras.models import load_model
from tensorflow.keras import models
from tensorflow.keras import backend
from tensorflow.keras import metrics as metrics_module

[MuhammedHasan]

Thanks for reporting the issue. I will fix the versioning issue.

[bomansonja]

Great thank you!

[MuhammedHasan]

I updated dependencies for mmsplice.Also, the latest numpy needs to be installed in colab because of cyvcf2 package so colab example updated to reflect that.