search

weizhepei / CasRel

A Novel Cascade Binary Tagging Framework for Relational Triple Extraction. Accepted by ACL 2020.
https://arxiv.org/abs/1909.03227
MIT License
763 stars 141 forks source link

IndexError: Read less bytes than requested #20

Open saarthak99 opened 4 years ago

saarthak99 commented 4 years ago

I get the following error while running the run.py file

!python run.py --train=True --dataset=NYT

Using TensorFlow backend.
2020-07-03 14:29:11.236282: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-07-03 14:29:11.241179: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2200000000 Hz
2020-07-03 14:29:11.241354: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1e66a00 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-03 14:29:11.241383: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-07-03 14:29:11.243340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-07-03 14:29:11.248062: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2020-07-03 14:29:11.248099: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: e54567bf18c0
2020-07-03 14:29:11.248114: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: e54567bf18c0
2020-07-03 14:29:11.248169: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 418.67.0
2020-07-03 14:29:11.248200: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 418.67.0
2020-07-03 14:29:11.248212: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 418.67.0
train_data len: 56195
dev_data len: 4999
test_data len: 1297
Traceback (most recent call last):
  File "run.py", line 40, in <module>
    subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels)
  File "/content/CasRel/model.py", line 15, in E2EModel
    bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None)
  File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 170, in load_trained_model_from_checkpoint
    load_model_weights_from_checkpoint(model, config, checkpoint_file, training=training)
  File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 114, in load_model_weights_from_checkpoint
    loader('bert/encoder/layer_%d/output/dense/kernel' % i),
  File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 18, in _loader
    return tf.train.load_variable(checkpoint_file, name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py", line 85, in load_variable
    return reader.get_tensor(name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/py_checkpoint_reader.py", line 70, in get_tensor
    self, compat.as_bytes(tensor_str))
IndexError: Read less bytes than requested

Please do let me know if there is any solution to this problem. Thanks in advance! :D

lizhenping commented 4 years ago

I get the following error while running the run.py file

!python run.py --train=True --dataset=NYT

Using TensorFlow backend.
2020-07-03 14:29:11.236282: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-07-03 14:29:11.241179: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2200000000 Hz
2020-07-03 14:29:11.241354: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1e66a00 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-03 14:29:11.241383: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-07-03 14:29:11.243340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-07-03 14:29:11.248062: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2020-07-03 14:29:11.248099: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: e54567bf18c0
2020-07-03 14:29:11.248114: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: e54567bf18c0
2020-07-03 14:29:11.248169: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 418.67.0
2020-07-03 14:29:11.248200: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 418.67.0
2020-07-03 14:29:11.248212: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 418.67.0
train_data len: 56195
dev_data len: 4999
test_data len: 1297
Traceback (most recent call last):
  File "run.py", line 40, in <module>
    subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels)
  File "/content/CasRel/model.py", line 15, in E2EModel
    bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None)
  File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 170, in load_trained_model_from_checkpoint
    load_model_weights_from_checkpoint(model, config, checkpoint_file, training=training)
  File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 114, in load_model_weights_from_checkpoint
    loader('bert/encoder/layer_%d/output/dense/kernel' % i),
  File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 18, in _loader
    return tf.train.load_variable(checkpoint_file, name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py", line 85, in load_variable
    return reader.get_tensor(name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/py_checkpoint_reader.py", line 70, in get_tensor
    self, compat.as_bytes(tensor_str))
IndexError: Read less bytes than requested

Please do let me know if there is any solution to this problem. Thanks in advance! :D

i have find how to change the code just annotation the os.device the the first issue

ChloeJKim commented 4 years ago

I had a similar issue (list index out of range) after running below:

python run.py --train=True --dataset=NYT

Using TensorFlow backend. WARNING:tensorflow:From run.py:11: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From run.py:13: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-09-08 20:36:29.369876: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-09-08 20:36:29.375942: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2399980000 Hz 2020-09-08 20:36:29.377320: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x555558c6f550 executing computations on platform Host. Devices: 2020-09-08 20:36:29.377352: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , train_data len: 56195 dev_data len: 4999 test_data len: 5000 WARNING:tensorflow:From /gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

Traceback (most recent call last): File "run.py", line 40, in subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels) File "/gstore/home/kimc26/CasRel/model.py", line 15, in E2EModel bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None) File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/loader.py", line 169, in load_trained_model_from_checkpoint kwargs) File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/loader.py", line 58, in build_model_from_config kwargs) File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/bert.py", line 84, in get_model dropout_rate=dropout_rate, File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/layers/embedding.py", line 37, in get_embedding )(inputs[0]), File "/gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/engine/base_layer.py", line 497, in call File "/gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/engine/base_layer.py", line 565, in _add_inbound_node IndexError: list index out of range

lizhenping commented 4 years ago

I had a similar issue (list index out of range) after running below:

python run.py --train=True --dataset=NYT

Using TensorFlow backend. WARNING:tensorflow:From run.py:11: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From run.py:13: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-09-08 20:36:29.369876: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-09-08 20:36:29.375942: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2399980000 Hz 2020-09-08 20:36:29.377320: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x555558c6f550 executing computations on platform Host. Devices: 2020-09-08 20:36:29.377352: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , train_data len: 56195 dev_data len: 4999 test_data len: 5000 WARNING:tensorflow:From /gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

Traceback (most recent call last): File "run.py", line 40, in subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels) File "/gstore/home/kimc26/CasRel/model.py", line 15, in E2EModel bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None) File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/loader.py", line 169, in load_trained_model_from_checkpoint kwargs) File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/loader.py", line 58, in build_model_from_config kwargs) File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/bert.py", line 84, in get_model dropout_rate=dropout_rate, File "/gstore/home/kimc26/.local/lib/python3.6/site-packages/keras_bert/layers/embedding.py", line 37, in get_embedding )(inputs[0]), File "/gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/engine/base_layer.py", line 497, in call File "/gstore/apps/Keras/2.2.4-foss-2017a-Python-3.6.3/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/engine/base_layer.py", line 565, in _add_inbound_node IndexError: list index out of range

find the code os.device and delete it and maker sure you use the gpu ,i just see CPU used in

weizhepei commented 4 years ago

Hi all, this seems like a preprocessing issue. You can check the discussion here to see if it helps. Please ping me here if there is still a problem with running the code.

exploreKK commented 2 years ago

Hi Guys, If anyone still have this issue, IndexError: Read less bytes than requested.. This is what worked for me,

My google drive disk ran out of memory and it was throwing me this error when I was trying to load model that I have previously saved in the google drive.

I used different google drive account with decent memory disk, to save and load model, and I had no issue.

Hope this helps!.

Thanks Kalpana

suexin27 commented 2 years ago

I had a similar issue (tensorflow.python.framework.errors_impl.OutOfRangeError: Read less bytes than requested) after running,Please do let me know if there is any solution to this problem. Traceback (most recent call last): File "run.py", line 40, in subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels) File "/home/suxin/CasRel-master/model.py", line 15, in E2EModel bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None) File "/home/suxin/anaconda3/envs/casrel/lib/python3.7/site-packages/keras_bert/loader.py", line 170, in load_trained_model_from_checkpoint load_model_weights_from_checkpoint(model, config, checkpoint_file, training=training) File "/home/suxin/anaconda3/envs/casrel/lib/python3.7/site-packages/keras_bert/loader.py", line 112, in load_model_weights_fromcheckpoint loader('bert/encoder/layer%d/intermediate/dense/kernel' % i), File "/home/suxin/anaconda3/envs/casrel/lib/python3.7/site-packages/keras_bert/loader.py", line 18, in _loader return tf.train.load_variable(checkpoint_file, name) File "/home/suxin/anaconda3/envs/casrel/lib/python3.7/site-packages/tensorflow/python/training/checkpoint_utils.py", line 82, in load_variable return reader.get_tensor(name) File "/home/suxin/anaconda3/envs/casrel/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 370, in get_tensor status) File "/home/suxin/anaconda3/envs/casrel/lib/python3.7/site-packages/tensorflow/python/framework/errors_impl.py", line 528, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.OutOfRangeError: Read less bytes than requested

exploreKK commented 2 years ago

@suexin27

If you are working in Google Colab, please try running with different account account, as the current one might have ran out of memory.

NGC2237-wei commented 2 years ago

I get the following error while running the run.py file !python run.py --train=True --dataset=NYT

Using TensorFlow backend.
2020-07-03 14:29:11.236282: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-07-03 14:29:11.241179: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 2200000000 Hz
2020-07-03 14:29:11.241354: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1e66a00 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-03 14:29:11.241383: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-07-03 14:29:11.243340: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-07-03 14:29:11.248062: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2020-07-03 14:29:11.248099: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: e54567bf18c0
2020-07-03 14:29:11.248114: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: e54567bf18c0
2020-07-03 14:29:11.248169: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 418.67.0
2020-07-03 14:29:11.248200: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 418.67.0
2020-07-03 14:29:11.248212: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 418.67.0
train_data len: 56195
dev_data len: 4999
test_data len: 1297
Traceback (most recent call last):
  File "run.py", line 40, in <module>
    subject_model, object_model, hbt_model = E2EModel(bert_config_path, bert_checkpoint_path, LR, num_rels)
  File "/content/CasRel/model.py", line 15, in E2EModel
    bert_model = load_trained_model_from_checkpoint(bert_config_path, bert_checkpoint_path, seq_len=None)
  File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 170, in load_trained_model_from_checkpoint
    load_model_weights_from_checkpoint(model, config, checkpoint_file, training=training)
  File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 114, in load_model_weights_from_checkpoint
    loader('bert/encoder/layer_%d/output/dense/kernel' % i),
  File "/usr/local/lib/python3.6/dist-packages/keras_bert/loader.py", line 18, in _loader
    return tf.train.load_variable(checkpoint_file, name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/checkpoint_utils.py", line 85, in load_variable
    return reader.get_tensor(name)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/py_checkpoint_reader.py", line 70, in get_tensor
    self, compat.as_bytes(tensor_str))
IndexError: Read less bytes than requested

Please do let me know if there is any solution to this problem. Thanks in advance! :D

i have find how to change the code just annotation the os.device the the first issue

hello,i can't find the os.device,could you tell me where it is,thanks