InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 322944 values, but the requested shape has 1213824

System information

What is the top-level directory of the model you are using: models/research/deep_speech
Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes, just to create the CSV files
OS Platform and Distribution : Windows 10
TensorFlow installed from (source or binary): installed using conda
TensorFlow version (use command below): 1.13.1
Bazel version (if compiling from source): N/A
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce: python deep_speech.py

You can collect some of this information using our environment capture script:

Describe the problem

I am using the deepspeech model on german Tuda Data instead of Librispeech and I get that error that I don't know where it come from

Source code / logs

2019-04-10 12:28:24.351834: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2 I0410 12:28:24.354882 4724 deep_speech.py:219] Data preprocessing... I0410 12:28:24.370544 4724 dataset.py:130] Loading data set C:/Users/MaggieEzzat/Desktop/german-speechdata-package-v2.tar/german-speechdata-package-v2/dev.csv I0410 12:28:24.375787 4724 dataset.py:130] Loading data set C:/Users/MaggieEzzat/Desktop/german-speechdata-package-v2.tar/german-speechdata-package-v2/dev.csv

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons If you depend on functionality not listed there, please file an issue.

I0410 12:28:24.951553 4724 run_config.py:532] Initializing RunConfig with distribution strategies. I0410 12:28:24.951553 4724 estimator_training.py:166] Not using Distribute Coordinator. I0410 12:28:24.951553 4724 estimator.py:201] Using config: {'_model_dir': '/Users/MaggieEzzat/Desktop/german-speechdata-package-v2.tar/german-speechdata-package-v2/deep_speech_model/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true graph_options { rewrite_options { meta_optimizer_iterations: ONE } } , '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': <tensorflow.contrib.distribute.python.one_device_strategy.OneDeviceStrategy object at 0x00000223EF756470>, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x00000223EF7565C0>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1, '_distribute_coordinator_mode': None} W0410 12:28:24.951553 4724 tf_logging.py:161] 'cpuinfo' not imported. CPU info will not be logged. W0410 12:28:24.951553 4724 tf_logging.py:161] 'psutil' not imported. Memory info will not be logged. I0410 12:28:27.223680 4724 logger.py:151] Benchmark run: {'model_name': 'deep_speech', 'dataset': {'name': 'Tuda Data'}, 'machine_config': {'gpu_info': {'count': 0}}, 'test_id': None, 'run_date': '2019-04-10T10:28:24.951553Z', 'tensorflow_version': {'version': '1.13.1', 'git_hash': "b'unknown'"}, 'tensorflow_environment_variables': [], 'run_parameters': [{'name': 'batch_size', 'long_value': 128}, {'name': 'is_bidirectional', 'bool_value': 'True'}, {'name': 'rnn_hidden_layers', 'long_value': 5}, {'name': 'rnn_hidden_size', 'long_value': 800}, {'name': 'rnn_type', 'string_value': 'gru'}, {'name': 'train_epochs', 'long_value': 10}, {'name': 'use_bias', 'bool_value': 'True'}]} I0410 12:28:27.223680 4724 deep_speech.py:277] Starting a training cycle: 1/10 W0410 12:28:27.223680 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py:429: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version. Instructions for updating: tf.py_func is deprecated in TF V2. Instead, use tf.py_function, which takes a python function which manipulates tf eager tensors instead of numpy arrays. It's easy to convert a tf eager tensor to an ndarray (just call tensor.numpy()) but having access to eager tensors means tf.py_functions can use accelerators such as GPUs as well as being differentiable using a gradient tape.

W0410 12:28:27.302863 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py:1419: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. I0410 12:28:27.316789 4724 estimator.py:1111] Calling model_fn. W0410 12:28:27.317786 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech_model.py:87: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.conv2d instead. W0410 12:28:27.325055 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech_model.py:58: batch_normalization (from tensorflow.python.layers.normalization) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.batch_normalization instead. W0410 12:28:27.403198 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech_model.py:114: GRUCell.init (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version. Instructions for updating: This class is equivalent as tf.keras.layers.GRUCell, and will be replaced by that in Tensorflow 2.0. W0410 12:28:27.403198 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech_model.py:121: bidirectional_dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version. Instructions for updating: Please use keras.layers.Bidirectional(keras.layers.RNN(cell)), which is equivalent to this API W0410 12:28:27.403198 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\ops\rnn.py:443: dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version. Instructions for updating: Please use keras.layers.RNN(cell), which is equivalent to this API W0410 12:28:28.732362 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech_model.py:182: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.dense instead. W0410 12:28:28.763608 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech.py:67: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. W0410 12:28:28.779408 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech.py:69: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. W0410 12:28:28.880683 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\keras\backend.py:5056: to_int64 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. I0410 12:28:32.300626 4724 estimator.py:1113] Done calling model_fn. I0410 12:28:32.389389 4724 basic_session_run_hooks.py:527] Create CheckpointSaverHook. I0410 12:28:39.288351 4724 monitored_session.py:222] Graph was finalized. 2019-04-10 12:28:39.299964: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance. W0410 12:28:39.307862 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\training\saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to check for files with this prefix. I0410 12:28:39.311793 4724 saver.py:1270] Restoring parameters from /Users/MaggieEzzat/Desktop/german-speechdata-package-v2.tar/german-speechdata-package-v2/deep_speech_model/model.ckpt-0 W0410 12:28:40.479333 4724 deprecation.py:323] From C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\training\saver.py:1070: get_checkpoint_mtimes (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file utilities to get mtimes. I0410 12:28:40.629921 4724 session_manager.py:491] Running local_init_op. I0410 12:28:40.715938 4724 session_manager.py:493] Done running local_init_op. I0410 12:28:50.240816 4724 basic_session_run_hooks.py:594] Saving checkpoints for 0 into /Users/MaggieEzzat/Desktop/german-speechdata-package-v2.tar/german-speechdata-package-v2/deep_speech_model/model.ckpt. I0410 12:29:00.461195 4724 util.py:164] Initialize strategy Traceback (most recent call last): File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call return fn(*args) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 322944 values, but the requested shape has 1213824 [[{{node dense/Tensordot}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech.py", line 413, in absl_app.run(main) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\absl\app.py", line 300, in run _run_main(main, args) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\absl\app.py", line 251, in _run_main sys.exit(main(argv)) File "C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech.py", line 406, in main run_deep_speech(flags_obj) File "C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech.py", line 284, in run_deep_speech estimator.train(input_fn=input_fn_train, hooks=train_hooks) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 358, in train loss = self._train_model(input_fn, hooks, saving_listeners) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1122, in _train_model return self._train_model_distributed(input_fn, hooks, saving_listeners) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1185, in _train_model_distributed self._config._train_distribute, input_fn, hooks, saving_listeners) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1287, in _actual_train_model_distributed saving_listeners) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1407, in _train_with_estimatorspec , loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss]) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\training\monitored_session.py", line 676, in run run_metadata=run_metadata) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1171, in run run_metadata=run_metadata) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1270, in run raise six.reraise(original_exc_info) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\six.py", line 693, in reraise raise value File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1255, in run return self._sess.run(args, *kwargs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1327, in run run_metadata=run_metadata) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1091, in run return self._sess.run(args, **kwargs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\client\session.py", line 929, in run run_metadata_ptr) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run feed_dict_tensor, options, run_metadata) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run run_metadata) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 322944 values, but the requested shape has 1213824 [[node dense/Tensordot (defined at C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech_model.py:182) ]]

Caused by op 'dense/Tensordot', defined at: File "C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech.py", line 413, in absl_app.run(main) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\absl\app.py", line 300, in run _run_main(main, args) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\absl\app.py", line 251, in _run_main sys.exit(main(argv)) File "C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech.py", line 406, in main run_deep_speech(flags_obj) File "C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech.py", line 284, in run_deep_speech estimator.train(input_fn=input_fn_train, hooks=train_hooks) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 358, in train loss = self._train_model(input_fn, hooks, saving_listeners) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1122, in _train_model return self._train_model_distributed(input_fn, hooks, saving_listeners) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1185, in _train_model_distributed self._config._train_distribute, input_fn, hooks, saving_listeners) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1254, in _actual_train_model_distributed self.config)) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\distribute\distribute_lib.py", line 1199, in call_for_each_replica return self._call_for_each_replica(fn, args, kwargs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\contrib\distribute\python\one_device_strategy.py", line 144, in _call_for_each_replica return fn(args, kwargs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1112, in _call_model_fn model_fn_results = self._model_fn(features=features, kwargs) File "C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech.py", line 177, in model_fn logits = model(features, training=True) File "C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech_model.py", line 182, in call logits = tf.layers.dense(inputs, self.num_classes, use_bias=self.use_bias) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\util\deprecation.py", line 324, in new_func return func(args, kwargs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\layers\core.py", line 188, in dense return layer.apply(inputs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1227, in apply return self.call(inputs, *args, *kwargs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\layers\base.py", line 530, in call outputs = super(Layer, self).call(inputs, args, kwargs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 554, in call outputs = self.call(inputs, *args, *kwargs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\keras\layers\core.py", line 968, in call outputs = standard_ops.tensordot(inputs, self.kernel, [[rank - 1], [0]]) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\ops\math_ops.py", line 3590, in tensordot ab_matmul, array_ops.concat([a_free_dims, b_free_dims], 0), name=name) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 7178, in reshape "Reshape", tensor=tensor, shape=shape, name=name) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper op_def=op_def) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func return func(args, **kwargs) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op op_def=op_def) File "C:\Users\MaggieEzzat\Anaconda3\envs\tf_env\lib\site-packages\tensorflow\python\framework\ops.py", line 1801, in init self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 322944 values, but the requested shape has 1213824 [[node dense/Tensordot (defined at C:\Users\MaggieEzzat\Desktop\deep_speech\deep_speech_model.py:182) ]]

tensorflow / models