How to implement multiple inputs and multiple outputs?

[JINYU-FAN]

I am trying to build models with multiple inputs and multiple outputs. A small example is created as below:

class Net(nn.Module):
    def __init__(self):
        nn.Module.__init__(self)
        self.fc1 = nn.Linear(1200,40)
        self.fc2 = nn.Linear(660,40)
        self.fc3 = nn.Linear(1200,50)
        self.fc4 = nn.Linear(660,50)    

    def forward(self,x:torch.Tensor,y:torch.Tensor):
        x = x.view(-1,1200)
        y = y.view(-1, 660)
        x1 = self.fc1(x)
        y1 = self.fc2(y)
        out1 = x1 + y1
        x2 = self.fc3(x)
        y2 = self.fc4(y)
        out2 = x2 + y2
        return out1, out2

The model is calibrated with:

x = torch.randn(100,1,40,30)
y = torch.randn(100,1,33,20)

After the calibration and test, I visualised the model with Netron:

However, when I try to create the runner with the model with the following code:

graph = xir.Graph.deserialize("deploy.xmodel")
root_subgraph = graph.get_root_subgraph()
child_subgraphs = root_subgraph.toposort_child_subgraph()
dpu_subgraphs = [cs for cs in child_subgraphs if cs.has_attr("device") and cs.get_attr("device").upper() == "DPU"]
print('dpu_subgraphs:',dpu_subgraphs)
runner = vart.Runner.create_runner(dpu_subgraphs[0], 'run')
inputTensors = runner.get_input_tensors()
outputTensors = runner.get_output_tensors()
print('dpu_subgraph_1:')
print(inputTensors, outputTensors)
input_ndims = tuple(inputTensors[0].dims)
output_ndims = tuple(outputTensors[0].dims)
print('input:',input_ndims)
print('output:',output_ndims)
runner = vart.Runner.create_runner(dpu_subgraphs[1], 'run')
inputTensors = runner.get_input_tensors()
outputTensors = runner.get_output_tensors()
print('dpu_subgraph_2:')
print(inputTensors, outputTensors)
input_ndims = tuple(inputTensors[0].dims)
output_ndims = tuple(outputTensors[0].dims)
print('input:',input_ndims)
print('output:',output_ndims)

The result gives:

dpu_subgraphs: [<xir.Subgraph named 'subgraph_NetNet_76_new'>, <xir.Subgraph named 'subgraph_NetNet_72_new'>] Xmodel compiled with batchSize: 1 Xmodel compiled with batchSize: 1 dpu_subgraph_1: [<xir.Tensor named 'NetNet_input_1_reshaped'>] [<xir.Tensor named 'NetNet_76_fix_new'>] input: (4, 4, 6, 50) output: (4, 1, 1, 50) Xmodel compiled with batchSize: 1 Xmodel compiled with batchSize: 1 dpu_subgraph_2: [<xir.Tensor named 'Net__Net_input_1_reshaped'>] [<xir.Tensor named 'Net__Net_72_fix_new'>] input: (4, 4, 6, 50) output: (4, 1, 1, 40)

Strangely, there is only the input buffer for x (40,30), and I could not figure out how to get access to the input y (33,20).

[lishixlnx]

can you please try input_ndims = tuple(inputTensors[1].dims) print('input:',input_ndims)

[JINYU-FAN]

Hi, thank you for replying! I have changed the code to:

print('dpu_subgraph_1:')
print(inputTensors, outputTensors)
input_ndims = tuple(inputTensors[1].dims)
output_ndims = tuple(outputTensors[1].dims)
print('input:',input_ndims)
print('output:',output_ndims)
runner = vart.Runner.create_runner(dpu_subgraphs[1], 'run')
inputTensors = runner.get_input_tensors()
outputTensors = runner.get_output_tensors()
print('dpu_subgraph_2:')
print(inputTensors, outputTensors)
input_ndims = tuple(inputTensors[1].dims)
output_ndims = tuple(outputTensors[1].dims)
print('input:',input_ndims)
print('output:',output_ndims)

The result is:

The same problem also exists with dpu_subgraph2: I printed out the inputTensors: [<xir.Tensor named 'Net__Net_input_1_reshaped'>] There is only one element in the list, which means that the inputTensors[1] would raise an error.

[lishixlnx]

can you please save your xmodel to some place for downloading?

[JINYU-FAN]

can you please save your xmodel to some place for downloading?

multi.zip Hi， I have uploaded the complete code above.

[Malarius1999]

Not quite sure if this will fix your problem, but I think you can tell the compiler (vai_c) that the last to layers should be outputs: --options '{"output_ops": "nameOut1,nameOut2"}' (Don't put a space after the "," when listing the output names)

See also: https://docs.xilinx.com/r/2.5-English/ug1414-vitis-ai/VAI_C-Usage

[SuZhan0322]

Hi @JINYU-FAN , any update about this issue?