TypeError: Error converting shape to a TensorShape: Dimension value must be integer or None or have an __index__ method, got value '<function max_length at 0x0000014A91940A60>' with type '<class 'function'>'

[rmothukuru]

@sravyadavuluri1999, In order to expedite the trouble-shooting process, please provide a code snippet to reproduce the issue reported here. Thanks!

[blakelucey]

I'm experiencing a similar error in my Kaggle notebook which can be viewed here.

The link above will provide you with all of the code that I have written so far, as well as my dataset and the error that I am experiencing.

The error in question, in full, is:

Error in py_call_impl(callable, dots$args, dots$keywords): TypeError: Error converting shape to a TensorShape: Dimension value must be integer or None or have an __index__ method, got value '[624, 9]' with type '<class 'list'>'.

Detailed traceback: 
  File "/usr/local/share/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/training/tracking/base.py", line 457, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/usr/local/share/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/keras/engine/sequential.py", line 202, in add
    batch_shape=batch_shape, dtype=dtype, name=layer.name + '_input')
  File "/usr/local/share/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_layer.py", line 311, in Input
    input_layer = InputLayer(**input_layer_config)
  File "/usr/local/share/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_layer.py", line 160, in __init__
    ragged=ragged)
  File "/usr/local/share/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/keras/backend.py", line 1223, in placeholder
    x = array_ops.placeholder(dtype, shape=shape, name=name)
  File "/usr/local/share/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py", line 3100, in placeholder
    return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
  File "/usr/local/share/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 6807, in placeholder
    shape = _execute.make_shape(shape, "shape")
  File "/usr/local/share/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/eager/execute.py", line 213, in make_shape
    raise TypeError("Error converting %s to a TensorShape: %s." % (arg_name, e))

Traceback:

1. model %>% layer_batch_normalization(input_shape = input_shape) %>% 
 .     layer_dense(units = 256, activation = "relu") %>% layer_batch_normalization() %>% 
 .     layer_dropout(rate = 0.3) %>% layer_dense(units = 256, activation = "relu") %>% 
 .     layer_batch_normalization() %>% layer_dropout(rate = 0.3) %>% 
 .     layer_dense(units = 256, activation = "relu") %>% layer_batch_normalization() %>% 
 .     layer_dropout(rate = 0.3) %>% layer_dense(units = 1, activation = "sigmoid") %>% 
 .     compile(loss = "binary_crossentropy", optimizer = "adam", 
 .         metrics = c("accuracy"))
2. withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))
3. eval(quote(`_fseq`(`_lhs`)), env, env)
4. eval(quote(`_fseq`(`_lhs`)), env, env)
5. `_fseq`(`_lhs`)
6. freduce(value, `_function_list`)
7. function_list[[i]](value)
8. layer_batch_normalization(., input_shape = input_shape)
9. create_layer(keras$layers$BatchNormalization, object, list(axis = as.integer(axis), 
 .     momentum = momentum, epsilon = epsilon, center = center, 
 .     scale = scale, beta_initializer = beta_initializer, gamma_initializer = gamma_initializer, 
 .     moving_mean_initializer = moving_mean_initializer, moving_variance_initializer = moving_variance_initializer, 
 .     beta_regularizer = beta_regularizer, gamma_regularizer = gamma_regularizer, 
 .     beta_constraint = beta_constraint, gamma_constraint = gamma_constraint, 
 .     renorm = renorm, renorm_clipping = renorm_clipping, renorm_momentum = renorm_momentum, 
 .     fused = fused, input_shape = normalize_shape(input_shape), 
 .     batch_input_shape = normalize_shape(batch_input_shape), batch_size = as_nullable_integer(batch_size), 
 .     dtype = dtype, name = name, trainable = trainable, virtual_batch_size = as_nullable_integer(virtual_batch_size), 
 .     adjustment = adjustment, weights = weights))
10. compose_layer(object, layer)
11. compose_layer.keras.engine.sequential.Sequential(object, layer)
12. object$add(layer)
13. py_call_impl(callable, dots$args, dots$keywords)

[zoyahav]

Thanks, however I'm not seeing any reference to tensorflow-transform either in the stacktrace or the linked notebook. Could you please clarify how this issue relates to TFT?

[sravyadavuluri1999]

@sravyadavuluri1999, In order to expedite the trouble-shooting process, please provide a code snippet to reproduce the issue reported here. Thanks! Thanks, however I'm not seeing any reference to tensorflow-transform either in the stacktrace or the linked notebook. Could you please clarify how this issue relates to TFT?

`def max_lengthTEMP(descriptions): lines = to_lines(descriptions)

max_length = np.max([len(text) for text in lines])
print(maxlen)

def data_generator(descriptions, photos, tokenizer, max_length): while 1: for key, description_list in descriptions.items(): photo = photos[key][0] input_image, input_sequence, output_word = create_sequences(tokenizer, max_length, description_list, photo) yield [[input_image, input_sequence], output_word] def create_sequences(tokenizer, max_length, desc_list, photo): X1, X2, y = list(), list(), list() for key, desc_list in descriptions.items(): for desc in desc_list: seq = tokenizer.texts_to_sequences([desc])[0] for i in range(1, len(seq)): in_seq, out_seq = seq[:i], seq[i] in_seq = pad_sequences([in_seq], maxlen=max_length)[0] out_seq= to_categorical([out_seq], num_classes=vocab_size)[0] X1.append(photo) X2.append(in_seq) y.append(out_seq) return array(X1),array(X2),array(y) from keras.utils import plot_model def define_model(vocab_size, max_length): inputs1 = Input(shape=(4096,)) fe1= Dropout(0.5)(inputs1) fe2= Dense(256, activation = 'relu')(fe1) inputs2 = Input(shape=(max_length,)) se1= Embedding(vocab_size, 256, mark_zero=True)(inputs2) se2= Dropout(0.5)(se1) se3= LSTM(256)(se2) decoder1=add([fe2, se3]) decoder2= Dense(256, activation='relu')(decoder1) outputs= Dense(vocab_size, activation='softmax')(decoder2)

model =Model(inputs=[inputs1, inputs2], outputs=outputs)
model.compile(loss='categorical_crossentropy', optimizer='adam')

print(model.summary())
plot_model(model, to_file='model.png', show_shapes=True)
return model

train_descriptions = load_clean_descriptions(dataset_models + "/" +'descriptions.txt',train) print('Descriptions: train=', len(train_descriptions)) train_features = load_photo_features(dataset_models + "/" +'features.pkl',train) print('Photos: train=', len(train_features))

model = define_model(vocab_size,max_length) epochs = 10 steps = len(train_descriptions)

for i in range(epochs): generator = data_generator(train_descriptions, train_features, tokenizer, max_length) model.fit_generator(generator, epochs=1, steps_perepoch=steps, verbose=1) model.save('model' + str(i) + '.h5')` i was trying to create model.h5 file then this error is showing

[zoyahav]

@sravyadavuluri1999 it's still unclear to me how this issue is related to tensorflow-transform, could you please clarify?

On the specific issue here:

Dimension value must be integer or None or have an index method, got value '[624, 9]' with type '<class 'list'>'.

Somewhere in the code something is trying to initialize a dimension with a list which is invalid (a dimension should be initialized with just its size).

[arghyaganguly]

Closing this due to inactivity.Please feel free to reopen with the requested information.

[SAFI36]

Bonjour, j'ai ce meme probleme dans mon code Si jeux avoir une repose rapidement

---

TypeError Traceback (most recent call last) ~\AppData\Local\Temp/ipykernel_6816/206995013.py in 11 batch_size=5, 12 shuffle=False, ---> 13 callbacks=callbacks 14 )

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs) 817 max_queue_size=max_queue_size, 818 workers=workers, --> 819 use_multiprocessing=use_multiprocessing) 820 821 def evaluate(self,

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py in fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs) 233 max_queue_size=max_queue_size, 234 workers=workers, --> 235 use_multiprocessing=use_multiprocessing) 236 237 total_samples = _get_total_number_of_samples(training_data_adapter)

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py in _process_training_inputs(model, x, y, batch_size, epochs, sample_weights, class_weights, steps_per_epoch, validation_split, validation_data, validation_steps, shuffle, distribution_strategy, max_queue_size, workers, use_multiprocessing) 591 max_queue_size=max_queue_size, 592 workers=workers, --> 593 use_multiprocessing=use_multiprocessing) 594 val_adapter = None 595 if validation_data:

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py in _process_inputs(model, mode, x, y, batch_size, epochs, sample_weights, class_weights, shuffle, steps, distribution_strategy, max_queue_size, workers, use_multiprocessing) 644 standardize_function = None 645 x, y, sample_weights = standardize( --> 646 x, y, sample_weight=sample_weights) 647 elif adapter_cls is data_adapter.ListsOfScalarsDataAdapter: 648 standardize_function = standardize

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, batch_size, check_steps, steps_name, steps, validation_split, shuffle, extract_tensors_from_dataset) 2344 # First, we build the model on the fly if necessary. 2345 if not self.inputs: -> 2346 all_inputs, y_input, dict_inputs = self._build_model_with_inputs(x, y) 2347 is_build_called = True 2348 else:

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\training.py in _build_model_with_inputs(self, inputs, targets) 2570 else: 2571 cast_inputs = inputs -> 2572 self._set_inputs(cast_inputs) 2573 return processed_inputs, targets, is_dict_inputs 2574

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\training.py in _set_inputs(self, inputs, outputs, training) 2657 kwargs['training'] = training 2658 try: -> 2659 outputs = self(inputs, **kwargs) 2660 except NotImplementedError: 2661 # This Model or a submodel is dynamic and hasn't overridden

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in call(self, inputs, *args, *kwargs) 771 not base_layer_utils.is_in_eager_or_tf_function()): 772 with auto_control_deps.AutomaticControlDependencies() as acd: --> 773 outputs = call_fn(cast_inputs, args, **kwargs) 774 # Wrap Tensors in outputs in tf.identity to avoid 775 # circular dependencies.

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\sequential.py in call(self, inputs, training, mask) 279 kwargs['training'] = training 280 --> 281 outputs = layer(inputs, **kwargs) 282 283 # outputs will be the inputs to the next layer.

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\layers\recurrent.py in call(self, inputs, initial_state, constants, kwargs) 642 643 if initial_state is None and constants is None: --> 644 return super(RNN, self).call(inputs, kwargs) 645 646 # If any of initial_state or constants are specified and are Keras

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in call(self, inputs, *args, **kwargs) 746 # Build layer if applicable (if the build method has been 747 # overridden). --> 748 self._maybe_build(inputs) 749 cast_inputs = self._maybe_cast_inputs(inputs) 750

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in _maybe_build(self, inputs) 2114 # operations. 2115 with tf_utils.maybe_init_scope(self): -> 2116 self.build(input_shapes) 2117 # We must set self.built since user defined build functions are not 2118 # constrained to set self.built.

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\layers\recurrent.py in build(self, input_shape) 562 if isinstance(self.cell, Layer): 563 if not self.cell.built: --> 564 self.cell.build(step_input_shape) 565 566 # set or validate state_spec

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\utils\tf_utils.py in wrapper(instance, input_shape) 304 if input_shape is not None: 305 input_shape = convert_shapes(input_shape, to_tuples=True) --> 306 output_shape = fn(instance, input_shape) 307 # Return shapes from fn as TensorShapes. 308 if output_shape is not None:

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\layers\recurrent.py in build(self, input_shape) 2299 regularizer=self.kernel_regularizer, 2300 constraint=self.kernel_constraint, -> 2301 caching_device=default_caching_device) 2302 self.recurrent_kernel = self.add_weight( 2303 shape=(self.units, self.units * 4),

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint, partitioner, use_resource, synchronization, aggregation, **kwargs) 444 synchronization=synchronization, 445 aggregation=aggregation, --> 446 caching_device=caching_device) 447 backend.track_variable(variable) 448

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\training\tracking\base.py in _add_variable_with_custom_getter(self, name, shape, dtype, initializer, getter, overwrite, kwargs_for_getter) 742 dtype=dtype, 743 initializer=initializer, --> 744 kwargs_for_getter) 745 746 # If we set an initializer and the variable processed it, tracking will not

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\keras\engine\base_layer_utils.py in make_variable(name, shape, dtype, initializer, trainable, caching_device, validate_shape, constraint, use_resource, collections, synchronization, aggregation, partitioner) 127 # TODO(apassos,rohanj) figure out how to remove collections from here so we 128 # can remove the V1. --> 129 variable_shape = tensor_shape.TensorShape(shape) 130 return tf_variables.VariableV1( 131 initial_value=init_val,

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\framework\tensor_shape.py in init(self, dims) 769 self._dims = [as_dimension(dims)] 770 else: --> 771 self._dims = [as_dimension(d) for d in dims_iter] 772 773 @property

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\framework\tensor_shape.py in (.0) 769 self._dims = [as_dimension(dims)] 770 else: --> 771 self._dims = [as_dimension(d) for d in dims_iter] 772 773 @property

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\framework\tensor_shape.py in as_dimension(value) 714 return value 715 else: --> 716 return Dimension(value) 717 718

~\anaconda3\envs\deeplearning\lib\site-packages\tensorflow_core\python\framework\tensor_shape.py in init(self, value) 198 TypeError("Dimension value must be integer or None or have " 199 "an index method, got {!r}".format(value)), --> 200 None) 201 if self._value < 0: 202 raise ValueError("Dimension %d must be >= 0" % self._value)

~\anaconda3\envs\deeplearning\lib\site-packages\six.py in raise_from(value, from_value)

TypeError: Dimension value must be integer or None or have an index method, got '1000100010001000'

[jean-moorman]

Thanks for nothign