How to add empty input to a node

[wjj19950828]

hi~Now I am adding a fuse pass

Now I have encountered a problem, how to add an empty input to a node, Node->addInput seems to be unable to add an empty input

Call the following api in python

node_def = helper.make_node(
    'Attention', # node name
    ['p2o.Add.6', 'qkv_weights', 'qkv_bias', '', '', 'extra_add_qk'],
    # ['p2o.Add.6', 'qkv_weights', 'qkv_bias', 'mask_index'],
    ['Y'], # outputs
    name='Attention',
    domain="com.microsoft",
    # attributes
    num_heads=9,
    unidirectional=0,
    )

You can create an Attention node with empty fourth and fifth inputs：

How to create it in C++，thanks a lot~

[wjj19950828]

@daquexian Do you have any good suggestions～

[HSQ79815]

@wjj19950828 you can try it following

Tensor empty_tensor;
empty_tensor.sizes().push_back(0); // make the tensor dims to [0]
empty_tensor.elem_type() =  ONNX_NAMESPACE::TensorProto_DataType_FLOAT ;  // you should modify it  as needed
Value *empty_value = graph.addInitializerAndCreateValue(empty_tensor);  // create a value object

node->addInput(empty_value)  // add input to node in order

[wjj19950828]

@HSQ79815 thank you for your reply～

my code as follow

Tensor mask_index, past;

mask_index.sizes().push_back(0); // Add mask_index empty node
mask_index.elem_type() = TensorProto_DataType_INT32;
past.sizes().push_back(0); // Add mask_index empty node
past.elem_type() = TensorProto_DataType_FLOAT;
Value *attention_inputs_mask_index = graph.addInitializerAndCreateValue(mask_index);
Value *attention_inputs_past = graph.addInitializerAndCreateValue(past);

attention_node->addInput(attention_inputs_mask_index);
attention_node->addInput(attention_inputs_past);

The generated Attention Node is as shown below

But when I run forward with ORT, the following error will be reported

The specific reason is because the code here causes

It means that past tensor does not consider it to be nullptr in ORT

But in the following Attention Node ORT can run

Do you know how I should modify,thanks a lot~

[HSQ79815]

@wjj19950828 are you able to share your model ? I try to reproduce

[wjj19950828]

@HSQ79815 The reproduction script and the onnx model are at the following link

Attention.onnx is the onnx model of a single Attention node generated by a python script, which can be run by onnx_run.py

ner_model_test_0706.onnxis the onnx model I generated by adding the attention fuse pass, the empty node is generated by the above code, but running ORT will report the above error

链接: https://pan.baidu.com/s/1bN-1VgJ9I8CxpHfDQxrQhg?pwd=mwhh 提取码: mwhh

感谢～

[HSQ79815]

@wjj19950828 Great, I will debug it sooner

[HSQ79815]

@wjj19950828

auto* empty_node = g->create(kUndefined, 1);
empty_node->output()->setUniqueName(""); // empty_node->output() is a `Value` that represents a empty tensor 

empty_node->insertBefore(node); // insert empty_node to node lists that should be topo sort

node->addInput(empty_node->output())

Are you able to use the above codes to verifying again ?

[wjj19950828]

@HSQ79815 thanks a lot~

The above code solved my problem