Xilinx / finn

Dataflow compiler for QNN inference on FPGAs
https://xilinx.github.io/finn
BSD 3-Clause "New" or "Revised" License
708 stars 225 forks source link

Need Transpose Before Flatten Layer 1D model Training #807

Open smtalds opened 1 year ago

smtalds commented 1 year ago

I applied this transform : # model = ModelWrapper(build_dir + "/end2end_cnv_w1a1_pre_post.onnx") model = model.transform(MoveScalarLinearPastInvariants()) model = model.transform(Streamline()) model = model.transform(LowerConvsToMatMul()) model = model.transform(MakeMaxPoolNHWC()) model = model.transform(absorb.AbsorbTransposeIntoMultiThreshold()) model = model.transform(absorb.AbsorbConsecutiveTransposes()) model = model.transform(ConvertBipolarMatMulToXnorPopcount()) model = model.transform(Streamline()) model = model.transform(absorb.AbsorbScalarMulAddIntoTopK()) model = model.transform(InferDataLayouts()) model = model.transform(RemoveUnusedTensors()) model.save(build_dir + "/end2end_cnv_w1a1_streamlined.onnx") And my onnx output this :

image #

Then i applied hls conversion :

mem_mode = "decoupled" model = ModelWrapper(build_dir + "/end2end_cnv_w1a1_streamlined.onnx") model = model.transform(to_hls.InferBinaryMatrixVectorActivation(mem_mode)) model = model.transform(to_hls.InferQuantizedMatrixVectorActivation(mem_mode)) model = model.transform(to_hls.InferLabelSelectLayer()) model = model.transform(to_hls.InferThresholdingLayer()) model = model.transform(to_hls.InferConvInpGen()) model = model.transform(MakeMaxPoolNHWC()) model = model.transform(absorb.AbsorbConsecutiveTransposes()) model = model.transform(to_hls.InferStreamingMaxPool()) model = model.transform(RemoveCNVtoFCFlatten()) model = model.transform(absorb.AbsorbConsecutiveTransposes()) model = model.transform(InferDataLayouts()) parent_model = model.transform(CreateDataflowPartition()) parent_model.save(build_dir + "/end2end_cnv_w1a1_dataflow_parent.onnx") sdp_node = parent_model.get_nodes_by_op_type("StreamingDataflowPartition")[0] sdp_node = getCustomOp(sdp_node) dataflow_model_filename = sdp_node.get_nodeattr("model") dataflow_model = ModelWrapper(dataflow_model_filename) dataflow_model.save(build_dir + "/end2end_cnv_w1a1_dataflow_model.onnx")

And my onnx output this : image

I need transpose after maxpool_streaming layer for flatten , how can i do that ?