There does not seem to be support for multi-way connection structures

sansi-zhang commented 1 year ago

Prerequisites

Please make sure to check off these prerequisites before submitting a bug report.

[√] Test that the bug appears on the current version of the dev-branch. Make sure to include the commit hash of the commit you checked out.
[√] Check that the issue hasn't already been reported, by checking the currently open issues.
[√] If there are steps to reproduce the problem, make sure to write them down below.
[√] If relevant, please include the ONNX files, which were created directly before and/or after the bug.

Quick summary

I had an error trying to do a multi-branch convolution.

Details

In my network, after the main branch is convolved, I need to convolve multiple branches separately, and then merge the results of the convolution using the concat operation (this step is not necessary).

Everything from generating the onnx file to converting it to the hls node worked fine, but an error of "cycle-free graph violated: partition depends on itself " was reported when model.transform(CreateDataflowPartition()) was executed.

I did some qualitative tests to find out what caused my problem. I have tried the hls node processing flow of Resnet-50 and conducted various tests, please refer to https://github.com/Xilinx/finn/issues/863#issue-1822075914 and https://github.com/Xilinx/finn/issues/860#issuecomment-1649853296 for details.

And I did a simple two-way connection test, but it still reported the same error, the onnx file is as follows:

Previously, the "add" node could be converted to "streamadd" normally, but it still reported an error.

I tried to modify the convolutional flow in the branch to keep only the convolution at the end of both branches (with bn and relu operations removed), but it still reported an error and even kept the "add" node.

Below is the onnx file for a test I did.

I tried to use the QuantIdentity() function to replicate the results of the upper convolutions for a multi-branch effect, and the add operation for the merge operation

I went on to try to use QuantIdentity() to replicate the results of the upper convolutional layer to achieve a multi-branch effect, using a convolutional path to process each branch, and using concat to merge the branches.

Unfortunately, the multiple approaches I tried stuck with model.transform(CreateDataflowPartition()), and they all gave the same error.

Therefore, I am here to ask if there is any reasonable way to achieve the operation of multiple branches.

Thanks again

Best wishes to you.

TATynise commented 10 months ago

Hi, I also have the same problem. I have tried the hls node processing flow of Resnet-50, but it doesn't work. How did you solve the multi-branch problem in the end?

sansi-zhang commented 10 months ago

Hi, I also have the same problem. I have tried the hls node processing flow of Resnet-50, but it doesn't work. How did you solve the multi-branch problem in the end?

I am sorry to tell you that I did not solve the multi-branch problem, I chose to adjust the network to avoid this phenomenon.

But my problem is multi-branch and double-chain branches, which should work fine for the normal resnet structure, you can refer to the processing steps in finn for details.

Xilinx / finn