Index out of bounds in a layer?

[stefanches7]

Hello, I run virnet on a dataset of 250k contigs with length from 100bp to 500kbp (pitily can not disclose the exact data), and get the following error:

Traceback (most recent call last):                                                                                                                                                                                   File "virnet/predict.py", line 48, in <module> 
                                                                                                                                                                      main()                                                                                                                                                                                                           
File "virnet/predict.py", line 44, in main                                                                                                                                                                           
predictions=run_pred(model,x_data)                                                                                                                                                                               
File "virnet/predict.py", line 19, in run_pred                                                                                                                                                                       
y_prop=model.predict(input_data)                                                                                                                                                                                 
File "/data/nasif12/home_if12/dvoretsk/projects/phagehost/false/virnet/virnet/NNClassifier.py", line 95, in predict 

return self.model.predict([X],batch_size=1024, verbose=1)                                                                                                                                                        
File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1462, in predict                                                                                                                        callbacks=callbacks)
                                                                                                                                                                                             File "/usr/local/lib/python3.6/dist-packages/keras/engine/training_arrays.py", line 324, in predict_loop 
                                                                                                            batch_outs = f(ins_batch)                                                                                                                                                                                        File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py", line 3292, in __call__   
                                                                                                          run_metadata=self.run_metadata)                                                                                                                                                                                  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1458, in __call__
                                                                                                            run_metadata_ptr)                                                                                                                                                                                              tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[184,599] = 3150 is not in [0, 3150)

         [[{{node embedding_1/embedding_lookup}}]]

(Sorry for ugly output, most infos are in the lower part) This issue looks quite similar to what I am getting. It may well be a layer inside the network that has too small dimension, but just as a suggestion to what it can be. I run python3 virnet/predict.py --input_dim=3000 --input=all_contigs_fragments.fasta --output=all_contigs_${rnd//0.}.csv, where rnd is a float between 0 and 1.

[stefanches7]

@alyosama

[alyosama]

Hi @stefanches7 I can not replicate your issue. Do you still have the same problem ? Could you send me a fake input or sample of your data so I can test that ?

[sxh1136]

@alyosama

Hi, I'm also getting a similar error:

Using TensorFlow backend. /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a futurstood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a futurstood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a futurstood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a futurstood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a futurstood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a futurstood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; inbe understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; inbe understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; inbe understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; inbe understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; inbe understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; inbe understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) Starting VirNet Loading Data /home/ubuntu/phageome/benchmark/fragments/test.fasta Traceback (most recent call last): File "predict.py", line 51, in main() File "predict.py", line 41, in main df_data=utils.load_data(args.input_path) File "/usr/local/software/virnet/utils.py", line 37, in load_data return load_fasta(input_path) File "/usr/local/software/virnet/utils.py", line 15, in load_fasta for record in SeqIO.parse(file_path, "fasta"): File "/usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/Bio/SeqIO/init.py", line 627, in parse i = iterator_generator(handle) File "/usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/Bio/SeqIO/FastaIO.py", line 181, in init super().init(source, alphabet=alphabet, mode="t", fmt="Fasta") File "/usr/local/software/conda/envs/virnet/lib/python3.6/site-packages/Bio/SeqIO/Interfaces.py", line 42, in init self.stream = open(source, "r" + mode) FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/phageome/benchmark/fragments/test.fasta'

This is the command I'm running: python3 predict.py --input_dim=3000 --input=/home/ubuntu/phageome/test.fasta --output=virnet_phage.csv

When i randomly subsample down the fasta however - to about 70 sequences, the script runs perfectly? sample.txt

[444thLiao]

Is it resolved? I also meet the same error when I using the wtp(what the phages). It is weird since it only got this problem in some genomes. Is anything wrongs? @alyosama