search

alyosama / virnet

VirNet: A deep attention model for viral reads identification
Apache License 2.0
30 stars 4 forks source link

what is the meaning of 'input dim' #8

Closed ucassee closed 2 years ago

ucassee commented 3 years ago

Hi,

I don't know what is the meaning of 'input dim'. My input file is the assembly of metagenomes, how should I set this parameter?

Thanks

alyosama commented 3 years ago

Input dim: is the maximum bp size in your data. The current tool can support " {100,500,1000 or 3000}. So if you have a maximum bp size of 2500, please use 3000. Otherwise, if you have reads more than 3000, you need to split it

ucassee commented 3 years ago

Hi @alyosama , Thanks for your reply.

If the longest contig is 8.9 kbp, should I split it into 3 sub-contigs and each sub-contig is shorter than 3kbp?

It is strange to restrict contigs to be shorter than 3000bp. The genome of the virus is often longer than 10 kbp.

alyosama commented 3 years ago

Hi @ucassee The tool is not designed for the assembly, it is for classification for HTS metagenomic reads. The goal is to purify these metagenomic reads from bacterial contaminations. The current technologies are generating reads with an average size of 500 bp or less.

So In your case, if you want to use our tool, you need to split your contigs to be shorter or equals to 3kbp.

ucassee commented 3 years ago

Okay, I get it, Thanks

ucassee commented 3 years ago

Hi @alyosama , I met error when using virnet in WTP(https://github.com/replikation/What_the_Phage), which integrates 12 viral identification tools. The shareware author of WTP said they split contigs to 3kbp and then import to virnet. I attached the error, could you please help me to debug it?

WARNING: underlay of /etc/localtime required more than 50 (93) bind mounts
Using TensorFlow backend.
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
WARNING: underlay of /etc/localtime required more than 50 (93) bind mounts
Using TensorFlow backend.
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.6/dist-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
2021-02-02 07:13:17.357673: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2021-02-02 07:13:17.416864: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1895280000 Hz
2021-02-02 07:13:17.431704: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5a62670 executing computations on platform Host. Devices:
2021-02-02 07:13:17.431779: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2021-02-02 07:13:17.580242: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

Starting VirNet
Loading Data TS01-B03_fragments.fasta
Loaded 8256 fragments
Loading Tokenizer
Start Predictions

1024/8256 [==>...........................] - ETA: 35s
2048/8256 [======>.......................] - ETA: 30sTraceback (most recent call last):
  File "/virnet/predict.py", line 51, in <module>
    main()
  File "/virnet/predict.py", line 47, in main
    predictions=run_pred(model,x_data)
  File "/virnet/predict.py", line 20, in run_pred
    y_prop=model.predict(input_data)
  File "/virnet/NNClassifier.py", line 100, in predict
    return self.model.predict([X],batch_size=1024, verbose=1)
  File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1462, in predict
    callbacks=callbacks)
  File "/usr/local/lib/python3.6/dist-packages/keras/engine/training_arrays.py", line 324, in predict_loop
    batch_outs = f(ins_batch)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py", line 3292, in __call__
    run_metadata=self.run_metadata)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1458, in __call__
    run_metadata_ptr)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[523,599] = 3150 is not in [0, 3150)
         [[{{node embedding_1/embedding_lookup}}]]

Thanks