ngraph custom operation cannot implement in self-built net.

[Maosquerade]

When I register a custom ngraph operation and use that in a self-build net as https://github.com/openvinotoolkit/openvino/tree/master/inference-engine/samples/ngraph_function_creation_sample and https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/samples/hello_reshape_ssd/reshape_ssd_extension.hpp. I found error like that:

terminate called after throwing an instance of 'InferenceEngine::details::InferenceEngineException'  
  what():  Unsupported primitive of type: CustomReLU name: CustomReLU_4 
Aborted (core dumped) 

and my build the net like this:

    auto arg0 = make_shared<opset1::Parameter>(element::f32, Shape{7});
    auto arg1 = make_shared<opset1::Parameter>(element::f32, Shape{7});
    auto arg2 = make_shared<opset1::Parameter>(element::f32, Shape{7});
    auto arg3 = make_shared<opset1::Parameter>(element::f32, Shape{7});

    auto relu = make_shared<CustomReluOp>(arg0);
    auto abs0 = make_shared<opset1::Abs>(relu);

    auto acos0 = make_shared<opset1::Acos>();

    auto add1 = shared_ptr<Node>(get_opset1().create("Add"));

    acos0->set_argument(0, relu);
    add1->set_argument(0, acos0);
    add1->set_argument(1, abs0);

    NodeVector ops{arg0, arg1, relu, abs0, acos0, add1};
    validate_nodes_and_infer_types(ops);

    auto ng_function = make_shared<Function>(OutputVector{add1}, ParameterVector{arg0, arg1});

the CustomReluOp is the custom operation in https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/samples/hello_reshape_ssd/reshape_ssd_extension.hpp then, I register the custom operation with

    Core ie;

    IExtensionPtr inPlaceExtension;
    if (device_name.find("CPU") != std::string::npos) {
        inPlaceExtension = std::make_shared<InPlaceExtension>();
        // register sample's custom kernel (CustomReLU)
        ie.AddExtension(inPlaceExtension);
    }

then error occured when I load the network:

    InferenceEngine::CNNNetwork net(ng_function);
    // prepare input data blop
    InferenceEngine::InputsDataMap inputInfo = net.getInputsInfo();

    for (auto& inputInfoItem: inputInfo) {
        inputInfoItem.second->setPrecision(Precision::FP32);
        inputInfoItem.second->setLayout(Layout::C);
    }
    // prepare output date blob
    string outputName;
    OutputsDataMap outputInfo = net.getOutputsInfo();
    for (auto& item: outputInfo) {
        DataPtr outputData = item.second;
        outputName = item.first;
        item.second->setPrecision(Precision::FP32);
        item.second->setLayout(Layout::C);
    }

    // Loading model to the device 
    std::cout << "Loading model to the device" << std::endl;
    InferenceEngine::ExecutableNetwork executable_network = ie.LoadNetwork(net, "CPU");

This error seems to be that the build-in net custom layer name will add ’_4' after a layer type？ Did this means a custom ngraph operation cannot implement in a built-in net ?

[ilya-lavrenov]

@ilyachur could you please help?

[ilyachur]

@Maosquerade CustomReLU_4 is the name of generated operation. The type of operation looks correct CustomReLU, but for some reason implementation for CPU wasn't loaded. I will take a look to this sample.

[ilyachur]

@Maosquerade Hi,

I tried to reproduce your issue. I changed the main.cpp file of hello_reshape_ssd sample to next view:

// Copyright (C) 2018-2020 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//

#include <vector>
#include <memory>
#include <string>

#include <ie_core.hpp>
#include <ngraph/ngraph.hpp>
#include <ngraph/opsets/opset1.hpp>

#include <samples/ocv_common.hpp>

#include "reshape_ssd_extension.hpp"

using namespace InferenceEngine;

int main(int argc, char* argv[]) {
    Core ie;

    IExtensionPtr inPlaceExtension;
    inPlaceExtension = std::make_shared<InPlaceExtension>();
    // register sample's custom kernel (CustomReLU)
    ie.AddExtension(inPlaceExtension);

    auto arg0 = std::make_shared<ngraph::opset1::Parameter>(ngraph::element::f32, ngraph::Shape{7});
    auto arg1 = std::make_shared<ngraph::opset1::Parameter>(ngraph::element::f32, ngraph::Shape{7});
    auto arg2 = std::make_shared<ngraph::opset1::Parameter>(ngraph::element::f32, ngraph::Shape{7});
    auto arg3 = std::make_shared<ngraph::opset1::Parameter>(ngraph::element::f32, ngraph::Shape{7});

    auto relu = std::make_shared<CustomReluOp>(arg0);
    auto abs0 = std::make_shared<ngraph::opset1::Abs>(relu);

    auto acos0 = std::make_shared<ngraph::opset1::Acos>();

    auto add1 = std::shared_ptr<ngraph::Node>(ngraph::get_opset1().create("Add"));

    acos0->set_argument(0, relu);
    add1->set_argument(0, acos0);
    add1->set_argument(1, abs0);

    ngraph::NodeVector ops{arg0, arg1, relu, abs0, acos0, add1};
    validate_nodes_and_infer_types(ops);

    auto ng_function = std::make_shared<ngraph::Function>(ngraph::OutputVector{add1}, ngraph::ParameterVector{arg0, arg1});

    CNNNetwork network(ng_function);

    ExecutableNetwork executable_network = ie.LoadNetwork(network, "CPU");
    return 0;
}

And executable network was successfully created. Can you please check this code?

[Maosquerade]

Thanks! I will check my code again!

[ilyachur]

@Maosquerade Do you have any update?

[Maosquerade]

@ilyachur After that I found the document of AddExtension and noticed that the function has another param to set device. Then I changed my code like:

ie.AddExtension(inPlaceExtension, "CPU");

and that worked! I just don't understand why that in my enviroment it has to add the param.

[ilyachur]

@Maosquerade It is strange. My code doesn't contain the second parameter and it works. Also both these method should load extension library to CPU plugin. Maybe do you use old version of the OpenVINO?

[AnastasiaKazantaeva]

It seems that the issues is not actual anymore. Closing it. Feel free to reopen it or create a new one,