multi-input-and-multi-output-models

[ghosh-inspire]

Hi, In this framework, is it possible to do multi output models. Something like the following: https://keras.io/getting-started/functional-api-guide/#multi-input-and-multi-output-models

If not, will you have any thoughts on how to implement it, which files and functions to dabble with. I am not able to understand, how to backpropagate the gradient. For example, while backproagating from two outputs, how do I combine the gradients at the point from where the network was split.

                        
[Input]-------**O**-------O-------O-------O-------O [Output-1]
                |
                |-------O-------O [Output-2]

[AlexeyAB]

@subhasish-ghosh Hi,

There is no way to make multiple inputs without code modification.

But you can use multiple outputs. Default model yolov3.cfg has 3 outputs - 3 [yolo] layers: https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov3.cfg

Branches are implemented by using [route] layers.

For example, while backproagating from two outputs, how do I combine the gradients at the point from where the network was split.

Just add deltas from both branches, as it is done in the [route] layer: https://github.com/AlexeyAB/darknet/blob/5a6afe96d3aa8aed19405577db7dba0ff173c848/src/route_layer.c#L97

[ghosh-inspire]

@AlexeyAB Thank you so much !!, I only need the multi-output option.

Can the outputs have different loss function. For example, Output-1 as a detector and Output-2 as a classifer ?

[alemelis]

hi @subhasish-ghosh any luck setting up this kind of net?

[ghosh-inspire]

For my experiment, I am using yolov3-tiny.cfg.

I was able to setup a network with a softmax after layer 9 and the rest of the network as is. I am training it with Pascal. The losses have decreased to 0.99. I printed the predictions from the softmax and its printing one of the classes from the bounding box in top-5, but the accuracies are not good. I need to train further maybe with COCO.

[alemelis]

how do you set the training?

can you train one softmax on different labels than the second softmax layer (like I'm trying in #3679)?

[ghosh-inspire]

Yes, I am having different labels for the detector and classifier but using the same dataset. I had to code it, it was not possible straightaway. Look in the file src/data.c. The classifier will finally land in the function: fill_truth(), you will have to modify this function to load the data as you like. You may also load the data like the detector fill_truth_detection(). Its just getting the labels from a txt file, instead of the file name.

[alemelis]

for future reference, I pushed my implementation here #3728