ConvolutionMethodHint ignored in SubGraph's?

[psyhtest]

I'm using dvdt-prof to profile OpenCL graph examples under Collective Knowledge.

To my surprise, for the SqueezeNet example, direct 1x1 convolution is only used 9 times for the squeeze1x1 layers, while GEMM-based convolution is used 16 times for the expand1x1 and expand3x3 layers (and once for the 7x7 conv1 layer). In other words, it appears that convolution_hint = ConvolutionMethodHint::DIRECT only gets applied to the convolutions directly added to the graph in main_graph_squeezenet(), but not to the convolutions indirectly added to the graph via get_expand_fire_node().

Could you confirm please?

[psyhtest]

Unfortunately, trying to force the hint for the subgraphs:

diff --git a/examples/graph_squeezenet.cpp b/examples/graph_squeezenet.cpp
index c8c411aa..a79bd80d 100644
--- a/examples/graph_squeezenet.cpp
+++ b/examples/graph_squeezenet.cpp
@@ -46,6 +46,7 @@ BranchLayer get_expand_fire_node(const std::string &data_path, std::string &&par
             get_weights_accessor(data_path, total_path + "expand1x1_w.npy"),
             get_weights_accessor(data_path, total_path + "expand1x1_b.npy"),
             PadStrideInfo(1, 1, 0, 0))
+        << ConvolutionMethodHint(ConvolutionMethodHint::DIRECT)
         << ActivationLayer(ActivationLayerInfo(ActivationLayerInfo::ActivationFunction::RELU));

     SubGraph i_b;

results in:

In file included from ./arm_compute/graph/nodes/BranchLayer.h:30:0,
                 from ./arm_compute/graph/Nodes.h:29,
                 from examples/graph_squeezenet.cpp:25:
./arm_compute/graph/SubGraph.h: In instantiation of ‘arm_compute::graph::SubGraph& arm_compute::graph::operator<<(arm_compute::graph::SubGraph&, Node) [with Node = arm_compute::graph::ConvolutionMethodHint]’:
examples/graph_squeezenet.cpp:49:63:   required from here
./arm_compute/graph/SubGraph.h:89:5: error: no matching function for call to ‘arm_compute::graph::SubGraph::add_node(arm_compute::support::cpp14::_Unique_if<arm_compute::graph::ConvolutionMethodHint>::_Single_object)’
     sub_graph.add_node(arm_compute::support::cpp14::make_unique<Node>(std::move(node)));
     ^~~~~~~~~
In file included from ./arm_compute/graph/nodes/BranchLayer.h:30:0,
                 from ./arm_compute/graph/Nodes.h:29,
                 from examples/graph_squeezenet.cpp:25:
./arm_compute/graph/SubGraph.h:51:10: note: candidate: void arm_compute::graph::SubGraph::add_node(std::unique_ptr<arm_compute::graph::INode>)
     void add_node(std::unique_ptr<INode> node);
          ^~~~~~~~
./arm_compute/graph/SubGraph.h:51:10: note:   no known conversion for argument 1 from ‘arm_compute::support::cpp14::_Unique_if<arm_compute::graph::ConvolutionMethodHint>::_Single_object {aka std::unique_ptr<arm_compute::graph::ConvolutionMethodHint, std::default_delete<arm_compute::graph::ConvolutionMethodHint> >}’ to ‘std::unique_ptr<arm_compute::graph::INode>’

[psyhtest]

For the GoogleNet example, direct 1x1 and 3x3 convolutions are only used once each, while GEMM-based convolutions are used 55 times.

[psyhtest]

For the MobileNet example, direct convolutions are not used, while GEMM-based convolutions are used 15 times.

In fact, there's even no hint in main_graph_mobilenet(), but the following patch:

diff --git a/examples/graph_mobilenet.cpp b/examples/graph_mobilenet.cpp
index 0c916c7b..0554917f 100644
--- a/examples/graph_mobilenet.cpp
+++ b/examples/graph_mobilenet.cpp
@@ -87,6 +87,7 @@ void main_graph_mobilenet(int argc, const char **argv)

     // Set target. 0 (NEON), 1 (OpenCL). By default it is NEON
     TargetHint target_hint = set_target_hint(argc > 1 ? std::strtol(argv[1], nullptr, 10) : 0);
+    ConvolutionMethodHint convolution_hint = target_hint == TargetHint::NEON ? ConvolutionMethodHint::GEMM : ConvolutionMethodHint::DIRECT;

     // Parse arguments
     if(argc < 2)
@@ -123,6 +124,7 @@ void main_graph_mobilenet(int argc, const char **argv)
     Graph graph;

     graph << target_hint
+          << convolution_hint
           << Tensor(TensorInfo(TensorShape(224U, 224U, 3U, 1U), 1, DataType::F32),
                     get_input_accessor(image, mean_r, mean_g, mean_b))
           << ConvolutionLayer(

again does not propagate the hint to the 1x1 convolutions added via get_dwsc_node().

[AnthonyBarbier]

Good catch! Indeed it looks like the hints are ignored in the subgraph, we'll have a look.

[AnthonyBarbier]

Fixed in 18.01