the installation test of pipeline-umi-amplicon failed

[dieunelderilus]

Hello- I have followed properly the instructions provided to install this pipeline-umi-amplicon. However, the installation test failed Command used: snakemake -j 1 -pr --configfile config.yml

Screenshot of the error message:

Please any help? ~DD

[philres]

Hi and thanks for reporting this.

Could you pleas try activating the conda environment and then running:

pip install numpy==1.19.5

and then try running:

snakemake -j 1 -pr --configfile config.yml

again?

Thanks, Philipp

[dieunelderilus]

Hello Philip,

Thank you for your response. Here is is the output of the pip intall command: (pipeline-umi-amplicon) [qlk5@biolinux pipeline-umi-amplicon]$ pip install numpy==1.19.5 _Requirement already satisfied: numpy==1.19.5 in /scicomp/home- pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages (1.19.5)_

Here is the output of of the snakemake -j 1 -pr --configfile config.yml command:

I have tried this in 2 different computers (cluster) but I still have the same error message at the end.

Any suggestions? Thanks in advance! ~DD

[philres]

HI again and sorry for not getting back to you earlier.

Could you please post the content of this file:

here?

Thanks, Philipp

[dieunelderilus]

Hello Philip - Thank you for your response, here is the content of the file smolecule.log file: <[08:51:22 - Smolecule] Given one input file, subreads are assumed to be grouped by read. [08:51:22 - Smolecule] Running spoa pre-medaka consensus for all reads. [08:51:23 - Smolecule] Writing medaka input bam for 1 reads. [08:51:23 - Smolecule] Running medaka consensus. [08:51:23 - Predict] Setting tensorflow inter/intra-op threads to 1/1. [08:51:23 - Predict] Processing region(s): 0:0-1206 [08:51:23 - Predict] Using model: /scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/data/r941_min_high_g360_model.hdf5. [08:51:23 - Predict] Processing 1 long region(s) with batching. [08:51:23 - ModelStore] filepath /scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/data/r941_min_high_g360_model.hdf5 [08:51:23 - ModelStore] ModelStore exception <class 'NotImplementedError'> Traceback (most recent call last): File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/bin/medaka", line 11, in sys.exit(main()) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/medaka.py", line 669, in main args.func(args) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/smolecule.py", line 443, in main medaka.prediction.predict(args) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/prediction.py", line 147, in predict model = model_store.load_model(time_steps=args.chunk_len) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/datastore.py", line 77, in load_model model = model_partial_function(time_steps=time_steps) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/models.py", line 146, in build_model model.add(Bidirectional(gru, input_shape=input_shape)) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 517, in _method_wrapper result = method(self, args, kwargs) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py", line 208, in add layer(x) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/wrappers.py", line 539, in call return super(Bidirectional, self).call(inputs, kwargs) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 951, in call return self._functional_construction_call(inputs, args, kwargs, File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1090, in _functional_construction_call outputs = self._keras_tensor_symbolic_call( File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 822, in _keras_tensor_symbolic_call return self._infer_output_signature(inputs, args, kwargs, input_masks) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 863, in _infer_output_signature outputs = call_fn(inputs, args, kwargs) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/wrappers.py", line 652, in call y = self.forward_layer(forward_inputs, File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 660, in call return super(RNN, self).call(inputs, kwargs) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1012, in call outputs = call_fn(inputs, args, kwargs) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent_v2.py", line 439, in call inputs, initialstate, = self._process_inputs(inputs, initial_state, None) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 859, in _process_inputs initial_state = self.get_initial_state(inputs) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 642, in get_initial_state init_state = get_initial_state_fn( File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 1948, in get_initial_state return _generate_zero_filled_state_for_cell(self, inputs, batch_size, dtype) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 2987, in _generate_zero_filled_state_for_cell return _generate_zero_filled_state(batch_size, cell.state_size, dtype) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 3005, in _generate_zero_filled_state return create_zeros(state_size) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 3000, in create_zeros return array_ops.zeros(init_state_size, dtype=dtype) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper return target(*args, *kwargs) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py", line 2819, in wrapped tensor = fun(args, kwargs) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py", line 2868, in zeros output = _constant_if_small(zero, shape, dtype, name) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py", line 2804, in _constant_if_small if np.prod(shape) < 1000: File "<__array_function__ internals>", line 5, in prod File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 3030, in prod return _wrapreduction(a, np.multiply, 'prod', axis, dtype, out, File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 87, in _wrapreduction return ufunc.reduce(obj, axis, dtype, out, **passkwargs) File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 852, in array raise NotImplementedError( NotImplementedError: Cannot convert a symbolic Tensor (bidirectional/forward_gru1/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported>

[KeelyDulmage]

Hello, just wanted to say that I had that same failed test error. I ran pip install numpy==1.19.5 before rerunning the test and it successfully fixed the issue on my end.

[dieunelderilus]

Hello @KeelyDulmage - Unfortunately I have tried your suggestion but it doesn't work in my computer. I have numpy==1.19.5 installed and running in my working environment. Thank you ~DD

[philres]

Hi again.

Thanks for sending through the log file. The error you get is the same i got with the wrong numpy version. When changing to the one we discussed above that problem was resolved.

I tried a few things, but I was unable to reproduce your problem which makes it very difficult to pinpoint the cause of the problem.

Could I ask you to run:

conda list > conda_env.log

and send me the log file please?

Thank you, Philipp

[dieunelderilus]

Hello Philip - Yes , this looks strange. Please find attached bellow the output of conda list > conda_env.log: conda_env.log

Thank you, ~DD

[scogi]

Same for me, installing numpy 1.19.5 did not resolve the problem (see also #13 )

[dieunelderilus]

Same for me, installing numpy 1.19.5 did not resolve the problem (see also #13 ) This is strange , they have closed this issue while it is still a serious problem for the user. I still don't know how to solve this. Cheers, ~DD

[passelma42]

Same for me, installing numpy 1.19.5 did not resolve the problem (see also #13 ) This is strange , they have closed this issue while it is still a serious problem for the user. I still don't know how to solve this. Cheers, ~DD

Just opened a new issue, pip install numpy==1.19.5 generates a numerous of dependency issues. Original versions in the environmen.yml file were not given. Building the conda env based on this file, will give you the latest versions of the predefined tools. Some of these tools need numpy 1.24.4 ... . Running the pipeline as it is, takes you as far as generating unpolished UMI consensuses and some stats... . Better than nothing, but I hope they will resolve the issues. (Or look for another tool which can handle UMI amplicons)