Understanding Data Generator

[vse-motec]

Dear all,

I have a question regarding the "return" values (actually: yield values) of the data generator for yolo3. In the file data.py, function "yolo3_data_generator" the yield statement is as follows: yield [image_data, *y_true], np.zeros(batch_size)

The documentation for a data generator that I found online suggests that the data generator should return/yield a pair of values x and y, where x is the data and y the corresponding annotation/ground truth labels.

When matching this documentation to the code, in my understanding the train data x becomes [image_data, *y_true] and the ground truth y becomes np.zeros(batch_size). However, this does not make sense to me. First, the data is a list with 2 elements which in my understanding are the data and the ground truth. Second, the second return value is just a bunch of zeros, which do not have a meaning.

Moreover, when playing around with the code and omitting the second value, i.e. changing the code to `yield [image_data, *y_true] completely breaks the training. There is no training progress, the total loss value is no longer the sum of the individual values and the resulting model produces garbage predictions. Therefore, the - in my eyes - meaningless second return value that only consists of zeros is the part that makes the training work in the first place.

Can anyone please shed light on this issue?

Thank you very munch in advance.

[david8862]

@vse-motec It is to match the loss calculating trick in this repo. We use a Lambda layer "yolo_loss" at the end of training model (here), with y_true & y_pred as input and loss value as output to get loss. And the final loss is just treated as the Lambda layer output, like:

    model.compile(optimizer=optimizer, loss={
        # use custom yolo_loss Lambda layer.
        'yolo_loss': lambda y_true, y_pred: y_pred})

[vse-motec]

Dear david8862,

thank you for taking the time to answer. Unfortunately, I am still not able to understand the exact situation here. Could you please elaborate more on the "loss calculating trick" you mentioned? What is the "trick" exactly in contrast e.g. to the normal expected way to calculate the loss.

Also I must admit you point to a part of code that I have additional questions about (see https://github.com/david8862/keras-YOLOv3-model-set/issues/123).

In case this is my lack of understanding python tricks or tensorflow internals please point me to some helpful documentation or give me some terms that help google. Because, e.g. just looking for lambda expressions in python is too general and did not help in this particular situation.

Thank you once again

[david8862]

@vse-motec a normal loss function in keras should be any callable with the signature loss_fn(y_true, y_pred) that returns an array of losses (see https://keras.io/api/losses/), but the (y_true, y_pred) param list is not enough for calculating yolo loss here (we need anchors array and some other options). So we

1. add an extra Lambda layer to use the normal (y_true, y_pred) and other related params as input, and the loss value tensors as output, then append this layer at the end of the train model.
2. use the loss tensor as model output (y_pred) and a np.zeros tensor as target label (y_true). Pass them to simple lambda loss function lambda y_true, y_pred: y_pred in model.compile

See following chart