rozerinyildiz
commented
2 years ago Thats the error message that i got. Can you pls suggest anything?
Closed rozerinyildiz closed 2 years ago
rozerinyildiz
commented
2 years ago Thats the error message that i got. Can you pls suggest anything?
gadagashwini
commented
2 years ago @rozerinyildiz, As the error suggests,
InvalidArgumentError Traceback (most recent call last) in () 5 validation_steps=len(valid_batches), 6 epochs=10, ----> 7 verbose=2
InvalidArgumentError, verbose =2 when used with ParameterServerStrategy .
Model.fit() function should be
model.fit(x=train_batches, steps_per_epoch=len(train_batches), validation_data=valid_batches, validation_steps=len(valid_batches), epochs=10, verbose=1 )
google-ml-butler[bot]
commented
2 years ago This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.
google-ml-butler[bot]
commented
2 years ago Closing as stale. Please reopen if you'd like to work on this further.
google-ml-butler[bot]
commented
2 years ago 
AnandaRauf
commented
2 years ago InvalidArgumentError Traceback (most recent call last)
InvalidArgumentError: Graph execution error:
gadagashwini
commented
2 years ago Hi @AnandaRauf, Could you share your complete code to investigate issue or raise new issue by providing all the information. Thank you!
AkanimohOD19A
commented
2 years ago Here's a similar challenge:
# Callbacks
callbacks = [ModelCheckpoint('.mdl_wts.hdf5', monitor='val_loss',mode='min',verbose=1, save_best_only=True),
ReduceLROnPlateau(monitor='val_loss', factor=0.3, patience=2, verbose=1, mode='min', min_lr=0.00000000001)]However, we the following is run
history = model.fit(datagen.flow(x_train, y_train, batch_size=42),validation_data = (x_val,y_val),epochs = 50,callbacks = callbacks)It throws error:
InvalidArgumentError Traceback (most recent call last)
1 frames /usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name) 53 ctx.ensure_initialized() 54 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, ---> 55 inputs, attrs, num_outputs) 56 except core._NotOkStatusException as e: 57 if name is not None:
InvalidArgumentError: Graph execution error:
HakanOzaydin
commented
2 years ago UnknownError Traceback (most recent call last)
meleayi
commented
1 year ago 
please support me ?
YameenV
commented
1 year ago Epoch 1/1000
747/879 [========================>.....] - ETA: 19:27 - loss: 2.5615 - acc: 0.5513
UnknownError Traceback (most recent call last) in 7 steps_per_epoch=steps_per_epoch, 8 validation_data=valid_generator, ----> 9 validation_steps=val_steps_per_epoch).history
1 frames /usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name) 53 ctx.ensure_initialized() 54 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, ---> 55 inputs, attrs, num_outputs) 56 except core._NotOkStatusException as e: 57 if name is not None:
UnknownError: Graph execution error:
2 root error(s) found. (0) UNKNOWN: OSError: image file is truncated (3 bytes not processed) Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/script_ops.py", line 270, in call ret = func(*args)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py", line 642, in wrapper return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 1030, in generator_py_func values = next(generator_state.get_iterator(iterator_id))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 831, in wrapped_generator for data in generator_fn():
File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 957, in generator_fn yield x[i]
File "/usr/local/lib/python3.7/dist-packages/keras/preprocessing/image.py", line 110, in getitem return self._get_batches_of_transformed_samples(index_array)
File "/usr/local/lib/python3.7/dist-packages/keras/preprocessing/image.py", line 342, in _get_batches_of_transformed_samples keep_aspect_ratio=self.keep_aspect_ratio)
File "/usr/local/lib/python3.7/dist-packages/keras/utils/image_utils.py", line 443, in load_img img = img.resize(width_height_tuple, resample)
File "/usr/local/lib/python3.7/dist-packages/PIL/Image.py", line 1886, in resize self.load()
File "/usr/local/lib/python3.7/dist-packages/PIL/ImageFile.py", line 247, in load "(%d bytes not processed)" % len(b)
OSError: image file is truncated (3 bytes not processed)
[[{{node PyFunc}}]] [[IteratorGetNext]] [[categorical_crossentropy/softmax_cross_entropy_with_logits/Shape_2/_6]](1) UNKNOWN: OSError: image file is truncated (3 bytes not processed) Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/script_ops.py", line 270, in call ret = func(*args)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py", line 642, in wrapper return func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 1030, in generator_py_func values = next(generator_state.get_iterator(iterator_id))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 831, in wrapped_generator for data in generator_fn():
File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 957, in generator_fn yield x[i]
File "/usr/local/lib/python3.7/dist-packages/keras/preprocessing/image.py", line 110, in getitem return self._get_batches_of_transformed_samples(index_array)
File "/usr/local/lib/python3.7/dist-packages/keras/preprocessing/image.py", line 342, in _get_batches_of_transformed_samples keep_aspect_ratio=self.keep_aspect_ratio)
File "/usr/local/lib/python3.7/dist-packages/keras/utils/image_utils.py", line 443, in load_img img = img.resize(width_height_tuple, resample)
File "/usr/local/lib/python3.7/dist-packages/PIL/Image.py", line 1886, in resize self.load()
File "/usr/local/lib/python3.7/dist-packages/PIL/ImageFile.py", line 247, in load "(%d bytes not processed)" % len(b)
OSError: image file is truncated (3 bytes not processed)
[[{{node PyFunc}}]] [[IteratorGetNext]]0 successful operations. 0 derived errors ignored. [Op:__inference_train_function_18084]
I often get this error
what will be the solution for this ??
abhinav52000
commented
12 months ago Everyone check once that you might have different class numbers in the dataset compared to what you have made a model with different numbers of classes in the dataset.
Safi2025
commented
6 months ago InvalidArgumentError Traceback (most recent call last) Input In [44], in <cell line: 1>() ----> 1 history = model.fit(train_generator, validation_data=validation_generator, epochs=50)
File ~\anaconda3\lib\site-packages\keras\src\utils\traceback_utils.py:70, in error_handler(*args, **kwargs)
60 @keras_export("keras.config.is_traceback_filtering_enabled")
61 def is_traceback_filtering_enabled():
62 """Check if traceback filtering is enabled.
63
64 Raw Keras tracebacks (also known as stack traces)
65 involve many internal frames, which can be
66 challenging to read through, while not being actionable for end users.
67 By default, Keras filters internal frames in most exceptions that it
68 raises, to keep traceback short, readable, and focused on what's
69 actionable for you (your own code).
---> 70
71 See also keras.config.enable_traceback_filtering() and
72 keras.config.disable_traceback_filtering().
73
74 If you have previously disabled traceback filtering via
75 keras.config.disable_traceback_filtering(), you can re-enable it via
76 keras.config.enable_traceback_filtering().
77
78 Returns:
79 Boolean, True if traceback filtering is enabled,
80 and False otherwise.
81 """
82 return global_state.get_global_attribute("traceback_filtering", True)
File ~\anaconda3\lib\site-packages\tensorflow\python\eager\execute.py:53, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name) 51 try: 52 ctx.ensure_initialized() ---> 53 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, 54 inputs, attrs, num_outputs) 55 except core._NotOkStatusException as e: 56 if name is not None:
InvalidArgumentError: Graph execution error:
Detected at node 'gradient_tape/binary_crossentropy/logistic_loss/mul/BroadcastGradientArgs' defined at (most recent call last):
File "C:\Users\GLOBAL SERVICE PLUS\anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\GLOBAL SERVICE PLUS\anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\GLOBAL SERVICE PLUS\anaconda3\lib\site-packages\ipykernel_launcher.py", line 16, in
** this is my code : history = model.fit(train_generator, validation_data=validation_generator, epochs=50)
arenran02
commented
5 months ago i have same problem when i set different output numbers for number of result classes. if you modify number of output layers, its error will be disappeared.
nooruzb
commented
5 months ago I am getting similar error: Input In [11], in <cell line: 12>() 6 layer.trainable = False 8 vgg_model.compile(loss='categorical_crossentropy', 9 optimizer=tf.keras.optimizers.SGD(momentum=0.9, learning_rate=0.001, decay=0.01), 10 metrics=['accuracy']) ---> 12 history = vgg_model.fit(train_generator, 13 epochs=30, 14 batch_size=64, 15 validation_data=test_generator, 16 callbacks=[early_stopping])
What could be wrong here in terms of the parameters?
I am using 128 classes for softmax vgg_model = tf.keras.Sequential(vgg_basemodel.layers[:-1]) vgg_model.add(tf.keras.layers.Dense(128, activation = 'softmax'))
The code is:
model.fit(x=train_batches, steps_per_epoch=len(train_batches), validation_data=valid_batches, validation_steps=len(valid_batches), epochs=10, verbose=2 )The error message is:
1 frames /usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs) 65 except Exception as e: # pylint: disable=broad-except 66 filtered_tb = _process_traceback_frames(e.traceback) ---> 67 raise e.with_traceback(filtered_tb) from None 68 finally: 69 del filtered_tb
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name) 53 ctx.ensure_initialized() 54 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, ---> 55 inputs, attrs, num_outputs) 56 except core._NotOkStatusException as e: 57 if name is not None:
InvalidArgumentError: Graph execution error:
Detected at node 'categorical_crossentropy/softmax_cross_entropy_with_logits' defined at (most recent call last): File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/usr/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/usr/local/lib/python3.7/dist-packages/traitlets/config/application.py", line 846, in launch_instance
app.start()
File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelapp.py", line 499, in start
self.io_loop.start()
File "/usr/local/lib/python3.7/dist-packages/tornado/platform/asyncio.py", line 132, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/usr/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, self._args)
File "/usr/local/lib/python3.7/dist-packages/tornado/platform/asyncio.py", line 122, in _handle_events
handler_func(fileobj, events)
File "/usr/local/lib/python3.7/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(args, kwargs)
File "/usr/local/lib/python3.7/dist-packages/zmq/eventloop/zmqstream.py", line 452, in _handle_events
self._handle_recv()
File "/usr/local/lib/python3.7/dist-packages/zmq/eventloop/zmqstream.py", line 481, in _handle_recv
self._run_callback(callback, msg)
File "/usr/local/lib/python3.7/dist-packages/zmq/eventloop/zmqstream.py", line 431, in _run_callback
callback(*args, *kwargs)
File "/usr/local/lib/python3.7/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(args, kwargs)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
handler(stream, idents, msg)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/ipkernel.py", line 208, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/zmqshell.py", line 537, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, *kwargs)
File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2718, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2828, in run_ast_nodes
if self.run_code(code, result):
File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 8, in
verbose=2
File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 64, in error_handler
return fn( args, kwargs)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1384, in fit
tmp_logs = self.train_function(iterator)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1021, in train_function
return step_function(self, iterator)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1010, in step_function
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1000, in run_step
outputs = model.train_step(data)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 860, in train_step
loss = self.compute_loss(x, y, y_pred, sample_weight)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 919, in compute_loss
y, y_pred, sample_weight, regularization_losses=self.losses)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/compile_utils.py", line 201, in call
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 141, in call
losses = call_fn(y_true, y_pred)
File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 245, in call
return ag_fn(y_true, y_pred, self._fn_kwargs)
File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 1790, in categorical_crossentropy
y_true, y_pred, from_logits=from_logits, axis=axis)
File "/usr/local/lib/python3.7/dist-packages/keras/backend.py", line 5099, in categorical_crossentropy
labels=target, logits=output, axis=axis)
Node: 'categorical_crossentropy/softmax_cross_entropy_with_logits'
logits and labels must be broadcastable: logits_size=[10,2] labels_size=[10,4]
[[{{node categorical_crossentropy/softmax_cross_entropy_with_logits}}]] [Op:__inference_train_function_716]