ParikhKadam / bidaf-keras

Bidirectional Attention Flow for Machine Comprehension implemented in Keras 2
GNU General Public License v3.0
64 stars 21 forks source link

Error while training #4

Closed KaminiSabu closed 5 years ago

KaminiSabu commented 5 years ago

Hello, nice efforts! and great help to me! :) I tried running this code without adding the pretrained model. I used command python3 -m bidaf --do_lowercase train --epochs=1 --steps_per_epoch=1 --validation_steps=1 But, this is leading to some errors while training. The terminal output after magnitude files are downloaded is given below. It says that the model is trained, but the accuracy is just 0.24 % which I suppose is because of the error. Can you please confirm? I am using tensorflow version 1.13.1

_WARNING:tensorflow:From bidaf-keras-master/python3env/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. WARNING:tensorflow:From bidaf-keras-master/python3env/lib/python3.6/site-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide. Epoch 1/1 2019-05-09 19:21:28.361366: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis node loss/combine_outputs_loss/strided_slice. Error: ValidateStridedSliceOp returned partial shapes [?,1,?] and [?,?] 2019-05-09 19:21:28.361456: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis node loss/combine_outputs_loss/strided_slice_1. Error: ValidateStridedSliceOp returned partial shapes [?,1,?] and [?,?] 2019-05-09 19:21:30.892571: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis node loss/combine_outputs_loss/strided_slice. Error: ValidateStridedSliceOp returned partial shapes [?,1,?] and [?,?] 2019-05-09 19:21:30.892620: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis node loss/combine_outputs_loss/strided_slice_1. Error: ValidateStridedSliceOp returned partial shapes [?,1,?] and [?,?] 2019-05-09 19:21:52.513339: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis node metrics/accuracy/strided_slice. Error: ValidateStridedSliceOp returned partial shapes [?,1,?] and [?,?] 2019-05-09 19:21:52.513381: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis node metrics/accuracy/strided_slice_1. Error: ValidateStridedSliceOp returned partial shapes [?,1,?] and [?,?] 2019-05-09 19:21:52.618175: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis node metrics/accuracy/strided_slice. Error: ValidateStridedSliceOp returned partial shapes [?,1,?] and [?,?] 2019-05-09 19:21:52.618211: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis node metrics/accuracy/strided_slice_1. Error: ValidateStridedSliceOp returned partial shapes [?,1,?] and [?,?] 1/1 [==============================] - 102s 102s/step - loss: 11.6590 - accuracy: 0.0029 - val_loss: 12.0245 - valaccuracy: 0.0024 Training Completed!

ParikhKadam commented 5 years ago

Hey.. Thank you...

It says that the model is trained, but the accuracy is just 0.24 % which I suppose is because of the error.

No.. this low accuracy isn't because of that error. You aren't training the model correctly. Try increasing the number of epochs and steps. And then look at the output you get. You will need to adjust batch_size according to your GPU size.

Now, about this error: 2019-05-09 19:21:52.618211: W ./tensorflow/core/grappler/optimizers/graph_optimizer_stage.h:241] Failed to run optimizer ArithmeticOptimizer, stage RemoveStackStridedSliceSameAxis node metrics/accuracy/strided_slice_1. Error: ValidateStridedSliceOp returned partial shapes [?,1,?] and [?,?]

It happens in tensorflow 1.13 and have no issues when run with tensorflow 1.12.0. Try running with that.. I have verified this error and hence closing the issue. If you still face problems, update here and I would reopen it.

Thank you..