out of range error

[myh10307]

While I am building a model using pre-trained Bert model, the following errors happen. There were no errors issued before the processing. I could successfully generate train and test data.

lay_num=12 model = load_trained_model_from_checkpoint(config_path, check_point_path, training=True, trainable=True, seq_len=Seq_len)

---

OutOfRangeError Traceback (most recent call last)

in () 1 lay_num=12 ----> 2 model = load_trained_model_from_checkpoint(config_path, check_point_path, training=True, trainable=True, seq_len=Seq_len) 4 frames /tensorflow-1.15.2/python3.6/tensorflow_core/python/pywrap_tensorflow_internal.py in get_tensor(self, tensor_str) 913 from tensorflow.python.util import compat 914 --> 915 return CheckpointReader_GetTensor(self, compat.as_bytes(tensor_str)) 916 917 __swig_destroy__ = _pywrap_tensorflow_internal.delete_CheckpointReader OutOfRangeError: Read less bytes than requested

[kimwoonggon]

I want to check where the error occurred Which file did you execute? Did you execute it on google Colab environment?

[myh10307]

Oh. I figured it out. It happened while I was modifying the vocab.txt. Thanks.

[kimwoonggon]

Great. I recommend that you execute the huggingface version that I had uploaded

[jaeuk412]

저도 같은 이슈인것 같은데요. 모델 out of range 관련해서 해결 방법을 찾아보았는데, 정확히 어떤 에러인지 몰라서 여기에 이슈 남겨봅니다ㅠ

print(config_path) print(checkpoint_path) print(SEQ_LEN) print(vocab_path)

bert/bert_config.json
bert/bert_model.ckpt
128
bert/vocab.txt

layer_num = 12 model = load_trained_model_from_checkpoint(config_path, checkpoint_path, training=True, trainable=True, output_layer_num=1, seq_len=SEQ_LEN,)

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-163-12333914aafd> in <module>()
      1 layer_num = 12
----> 2 model = load_trained_model_from_checkpoint(config_path, checkpoint_path, training=True, trainable=True, output_layer_num=1, seq_len=SEQ_LEN,)

5 frames
/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py in load_trained_model_from_checkpoint(config_file, checkpoint_file, training, trainable, output_layer_num, seq_len, **kwargs)
    167         trainable=trainable,
    168         output_layer_num=output_layer_num,
--> 169         seq_len=seq_len,
    170         **kwargs)
    171     load_model_weights_from_checkpoint(model, config, checkpoint_file, training=training)

/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py in build_model_from_config(config_file, training, trainable, output_layer_num, seq_len, **kwargs)
     56         trainable=trainable,
     57         output_layer_num=output_layer_num,
---> 58         **kwargs)
     59     if not training:
     60         inputs, outputs = model

/usr/local/lib/python3.6/dist-packages/keras_bert/bert.py in get_model(token_num, pos_num, seq_len, embed_dim, transformer_num, head_num, feed_forward_dim, dropout_rate, attention_activation, feed_forward_activation, training, trainable, output_layer_num, use_task_embed, task_num)
     82         embed_dim=embed_dim,
     83         pos_num=pos_num,
---> 84         dropout_rate=dropout_rate,
     85     )
     86     if use_task_embed:

/usr/local/lib/python3.6/dist-packages/keras_bert/layers/embedding.py in get_embedding(inputs, token_num, pos_num, embed_dim, dropout_rate, trainable)
     35             trainable=trainable,
     36             name='Embedding-Token',
---> 37         )(inputs[0]),
     38         keras.layers.Embedding(
     39             input_dim=2,

/tensorflow-1.15.2/python3.6/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)

/tensorflow-1.15.2/python3.6/keras/engine/base_layer.py in _add_inbound_node(self, input_tensors, output_tensors, input_masks, output_masks, input_shapes, output_shapes, arguments)

IndexError: list index out of range

[jaeuk412]

(03_케라스로_버트_빠르게_돌려보기_With_네이버_영화_감성분석_TUTORIAL.ipynb) tensorflow(1.15.2)가 keras(2.3.1)과 맞는것 같아 keras(2.3.1)로 별도 설치했는데. !pip install keras-bert 입력시, keras>=2.4.3 에러뜨면서 자동 keras(2.4.3)으로 변경됬습니다.

ERROR: keras-bert 0.86.0 has requirement Keras>=2.4.3, but you'll have keras 2.3.0 which is incompatible.

아마, keras(2.4.3)와 tensorflow(1.15.2) 호환이 안되서 났던 에러 같습니다. 현재 keras-bert(0.86.0), keras(2.4.3), tf(2.3.0) 으로 버전수정하니까 model Flow 확인하는 곳까지 도달 했는데요. 'get_bert_finetuning_model()' 여기서 다시 막혔습니다ㅠ

from IPython.display import SVG
from keras.utils.vis_utils import model_to_dot

# SVG(model_to_dot(model).create(prog='dot', format='svg'))
SVG(model_to_dot(get_bert_finetuning_model(model), dpi=65).create(prog='dot', format='svg'))

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-52-4aa880b2d989> in <module>()
      3 
      4 # SVG(model_to_dot(model).create(prog='dot', format='svg'))
----> 5 SVG(model_to_dot(get_bert_finetuning_model(model), dpi=65).create(prog='dot', format='svg'))

7 frames
<ipython-input-51-3a5c813dfba4> in get_bert_finetuning_model(model)
      4 
      5 
----> 6   outputs = keras.layers.Dense(1, activation='sigmoid',kernel_initializer=keras.initializers.TruncatedNormal(stddev=0.02),name = 'real_output')(dense)
      7 
      8   bert_model = keras.models.Model(inputs, outputs)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py in __call__(self, *args, **kwargs)
    774             try:
    775               with ops.enable_auto_cast_variables(self._compute_dtype_object):
--> 776                 outputs = call_fn(cast_inputs, *args, **kwargs)
    777 
    778             except errors.OperatorNotAllowedInGraphError as e:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/core.py in call(self, inputs)
   1196         self.bias,
   1197         self.activation,
-> 1198         dtype=self._compute_dtype_object)
   1199 
   1200   def compute_output_shape(self, input_shape):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/ops/core.py in dense(inputs, kernel, bias, activation, dtype)
     51       outputs = sparse_ops.sparse_tensor_dense_matmul(inputs, kernel)
     52     else:
---> 53       outputs = gen_math_ops.mat_mul(inputs, kernel)
     54   # Broadcast kernel to inputs.
     55   else:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_math_ops.py in mat_mul(a, b, transpose_a, transpose_b, name)
   5640   _, _, _op, _outputs = _op_def_library._apply_op_helper(
   5641         "MatMul", a=a, b=b, transpose_a=transpose_a, transpose_b=transpose_b,
-> 5642                   name=name)
   5643   _result = _outputs[:]
   5644   if _execute.must_record_gradient():

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(op_type_name, name, **keywords)
    307     # Need to flatten all the arguments into a list.
    308     # pylint: disable=protected-access
--> 309     g = ops._get_graph_from_inputs(_Flatten(keywords.values()))
    310     # pylint: enable=protected-access
    311   except AssertionError as e:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in _get_graph_from_inputs(op_input_list, graph)
   6111         graph = getattr(graph_element, "graph", None)
   6112       elif original_graph_element is not None:
-> 6113         _assert_same_graph(original_graph_element, graph_element)
   6114       elif graph_element.graph is not graph:
   6115         raise ValueError("%s is not from the passed-in graph." % graph_element)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in _assert_same_graph(original_item, item)
   6046     raise ValueError(
   6047         "%s must be from the same graph as %s (graphs are %s and %s)." %
-> 6048         (item, original_item, graph, original_graph))
   6049 
   6050 

ValueError: Tensor("real_output_4/kernel/Read/ReadVariableOp:0", shape=(768, 1), dtype=float32) must be from the same graph as Tensor("NSP-Dense/Tanh:0", shape=(?, 768), dtype=float32) (graphs are <tensorflow.python.framework.ops.Graph object at 0x7f2170e55ef0> and FuncGraph(name=keras_graph, id=139781576151280)).

[pooreun]

IndexError: list index out of range keras 버전 문제네요

!pip install keras-transformer==0.32.0 !pip install keras-bert==0.81.0 !pip install keras-radam==0.15.0

이러니 정상적으로 됩니다.