input-order-mismatch issue in multi-input layer.

[EunjuYang]

While I'm implementing an example app. I faced an issue of the order mismatch between addLayer( ) and input tensor (multi-input layer case). Here're some descriptions on this issue.

1. I added some layers to my model, named proto_net:

   // TODO: [Bugfix] sequence of addLayer does not match with the input order.
   proto_net->addLayer(input_major_label);
   proto_net->addLayer(input_feature);
   proto_net->addLayer(l2);    /** input_feature is input of l2 **/
   proto_net->addLayer(proto); /** input_major_label and l2 are inputs of protonet **/

   where the protonet consits of four layers; the last layer proto takes two input l2norm and input_major_label:

  LayerHandle proto = ml::train::createLayer(
    "protonet",
    {"name=protonet", "input_layers=[input_major_label, l2norm]",
     "maxnum_major_class=" + std::to_string(max_num_class),
     "maxnum_minor_class=" + std::to_string(max_num_class), "trainable=false"});

2. Different with my expectation, however, the actual graph's inputs do not follow the order. The input tensors' order does not matched with the addLayer order (protonet.cpp):

static constexpr size_t FIRST_INOUT_IDX = 0;
static constexpr size_t SECOND_INOUT_IDX = 1;

void Protonet::forwarding(nntrainer::RunLayerContext &context, bool training) {
  ...
  auto &features_ = context.getInput(FIRST_INOUT_IDX);
  auto &major_class_id = context.getInput(SECOND_INOUT_IDX);

This bugfix burden is passed to a user, which is required to be resolved.

(+) The order of input tensors of the layer is only affected by the order of addLayer. But it is not coherent with the order (affected but randomly decided).

(+) I tested with various combinations to boil down the isuse; I changed input_layers option in layer creation as well. The input_layers is irrelevant with the actual input tensors' order. The following code change works in the same situation:

  LayerHandle proto = ml::train::createLayer(
    "protonet",
    {"name=protonet", "input_layers=[l2norm, input_major_label]",
     "maxnum_major_class=" + std::to_string(max_num_class),
     "maxnum_minor_class=" + std::to_string(max_num_class), "trainable=false"});

---

Summary of Issue

• The order of input tensor seems to be defined by the order of addLayer when a model is created.
• However, the order is not coherent with it.
• No error or warning when the order of input_layers property does not match with the actual order.

Candidate to resolve

• It would be better to enhance our code to build the graph following the order of input_layers, not the order of addLayer.

[taos-ci]

:octocat: cibot: Thank you for posting issue #2660. The person in charge will reply soon.

[lhs8928]

This issue is related with issue #1126 and #1813.