Xilinx / finn

Dataflow compiler for QNN inference on FPGAs
https://xilinx.github.io/finn
BSD 3-Clause "New" or "Revised" License
708 stars 225 forks source link

AssertionError: Im2Col_0: ConvolutionInputGenerator1D works only for 1D convs #814

Open joshua99911 opened 1 year ago

joshua99911 commented 1 year ago

Discussed in https://github.com/Xilinx/finn/discussions/602

Originally posted by **sakshigoel155** May 27, 2022 Hi, I have a model with 4 2D convolution layers followed by 3 dense layers. I generate an ONNX model for it with input size (1,1,4,512). While trying out DataflowBuildConfig on it to generate a FINN representation, I get the following error. Final outputs will be generated in output_estimates_only Build log is at output_estimates_only/build_dataflow.log Running step: step_qonnx_to_finn [1/8] Running step: step_tidy_up [2/8] Running step: step_streamline [3/8] Traceback (most recent call last): File "/workspace/finn/src/finn/builder/build_dataflow.py", line 166, in build_dataflow_cfg model = transform_step(model, cfg) File "/workspace/finn/src/finn/builder/build_dataflow_steps.py", line 295, in step_convert_to_hls model = model.transform(to_hls.InferConvInpGen()) File "/workspace/finn-base/src/finn/core/modelwrapper.py", line 141, in transform (transformed_model, model_was_changed) = transformation.apply( File "/workspace/finn/src/finn/transformation/fpgadataflow/convert_to_hls_layers.py", line 201, in apply assert is_1d_convolution, ( AssertionError: Im2Col_0: ConvolutionInputGenerator1D works only for 1D convs ======================================================================= I also tried the flow using bnn-pynq tutorial and after the streamlining step, while converting to HLS with createdataflowpartition, I get the same error, ![error](https://user-images.githubusercontent.com/74159729/170526086-974ecf0f-d0ec-42e2-9d68-1e06e1244d8c.png) I looked at the error further and it seems, it can only work for square input images. My input is not square input image but a 4X512 input, which uses EEG time series data from 4 channels, 512 samples at a time. The graph after streamlining, just before parent_model = model.transform(CreateDataflowPartition()), without fpgadataflow.convert_to_hls_layers.InferConvInpGen, looks like in the attached image ![QAT-1_IR1 onnx](https://user-images.githubusercontent.com/74159729/170522042-00d654fe-69af-421a-9420-9afa77331a4d.png) Can someone please help in finding out a solution on how can i go ahead?. Will be really thankful if someone can help me, as my project deadline is near. Thanks
joshua99911 commented 1 year ago

To add some context, running into almost the exact same issue though did not want to repost as the discussion existed, so just turned it from a discussion to an issue.

auphelia commented 1 year ago

Hi @joshua99911 ,

Thanks for providing all this information, could you please convert the request by using the provided template: https://github.com/Xilinx/finn/blob/main/.github/ISSUE_TEMPLATE/bug_report.md