ValueError: operands could not be broadcast together with shapes (32,) (6,) (32,)

[algoteam5]

Hi, except NNCLR, other models faced this error ValueError: operands could not be broadcast together with shapes (32,) (6,) (32,)

Here is the log:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)
    106   def _method_wrapper(self, *args, **kwargs):
    107     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
--> 108       return method(self, *args, **kwargs)
    109 
    110     # Running inside `run_distribute_coordinator` already.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
   1101               logs = tmp_logs  # No error, now safe to assign to logs.
   1102               end_step = step + data_handler.step_increment
-> 1103               callbacks.on_train_batch_end(end_step, logs)
   1104         epoch_logs = copy.copy(logs)
   1105 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/callbacks.py in on_train_batch_end(self, batch, logs)
    438     """
    439     if self._should_call_train_batch_hooks:
--> 440       self._call_batch_hook(ModeKeys.TRAIN, 'end', batch, logs=logs)
    441 
    442   def on_test_batch_begin(self, batch, logs=None):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/callbacks.py in _call_batch_hook(self, mode, hook, batch, logs)
    287       self._call_batch_begin_hook(mode, batch, logs)
    288     elif hook == 'end':
--> 289       self._call_batch_end_hook(mode, batch, logs)
    290     else:
    291       raise ValueError('Unrecognized hook: {}'.format(hook))

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/callbacks.py in _call_batch_end_hook(self, mode, batch, logs)
    307       batch_time = time.time() - self._batch_start_time
    308 
--> 309     self._call_batch_hook_helper(hook_name, batch, logs)
    310 
    311     if self._check_timing:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/callbacks.py in _call_batch_hook_helper(self, hook_name, batch, logs)
    340       hook = getattr(callback, hook_name)
    341       if getattr(callback, '_supports_tf_logs', False):
--> 342         hook(batch, logs)
    343       else:
    344         if numpy_logs is None:  # Only convert once.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/callbacks.py in on_train_batch_end(self, batch, logs)
    959 
    960   def on_train_batch_end(self, batch, logs=None):
--> 961     self._batch_update_progbar(batch, logs)
    962 
    963   def on_test_batch_end(self, batch, logs=None):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/callbacks.py in _batch_update_progbar(self, batch, logs)
   1015       # Only block async when verbose = 1.
   1016       logs = tf_utils.to_numpy_or_python_type(logs)
-> 1017       self.progbar.update(self.seen, list(logs.items()), finalize=False)
   1018 
   1019   def _finalize_progbar(self, logs):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/generic_utils.py in update(self, current, values, finalize)
    556           self._values[k] = [v * value_base, value_base]
    557         else:
--> 558           self._values[k][0] += v * value_base
    559           self._values[k][1] += value_base
    560       else:

ValueError: operands could not be broadcast together with shapes (32,) (6,) (32,) 

[algoteam5]

please help @beresandras

[beresandras]

Hi! All except NNCLR? If I run the Colab notebook using any algorithm, it runs without an error. What changes did you make?

[algoteam5]

Yeah, all except NNCLR. I didn't change anything but my dataset. The thing is I already made sure my dataset format is similar to the original.

[beresandras]

Okay, but there are no hyperparameters with the value of either 32 or 6 in the code (which is in the error message). What is your image shape? What is the model's output size? How many classes do you have? Do you train on one GPU? Is NNCLR giving decent results?

Since the original code runs without an error, one of the changes must be causing it.

[algoteam5]

What is your image shape? -> 64x64 What is the model's output size? -> dim=128 How many classes do you have? -> 248 classes Do you train on one GPU? -> I tried both on one GPU and multiple GPUs, and both produced the same error. Is NNCLR giving decent results? -> not so good, kinda bad TBH

[beresandras]

Try to find what the error message refers to, what has the shape of (32,) and (6,).