microsoft / nnfusion

A flexible and efficient deep neural network (DNN) compiler that generates high-performance executable from a DNN model description.
MIT License
952 stars 158 forks source link

[BUG] nnfusion fail: Unknown op count: 2 #533

Open husterdjx opened 1 month ago

husterdjx commented 1 month ago

šŸ› Bug Hello, I'm trying to use nnfusion to gen cuda code for transformer-based model provided by hugging face. But after I get onnx format model by optimum-cli, I cannot deliver this onnx model to nnfusion correctly. I wonder whether I get the model right or nnfusion cannot smoothly transform these kind of transformer-based model?

To Reproduce Steps to reproduce the behavior:

  1. use optimum-cli to convert the hugging face checkpoint to onnx format model:
    optimum-cli export onnx --model google/vit-base-patch16-224-in21k vit-base-patch16-224-in21k_onnx_32/ --batch_size 32

    (--bath_size arg seems not change the onnx output when I visualize both by netron)

  2. use nnfusion to convert (ERROR)
    nnfusion ./vit-base-patch16-224-in21k_onnx_32/model.onnx -f onnx -p "batch_size:32;num_channels:3;height:224;width:224"

    and the error output meg:

    
    ...
    name: "main_graph"
    input {
    name: "pixel_values"
    type {
      tensor_type {
        elem_type: 1
        shape {
          dim {
            dim_param: "batch_size"
          }
          dim {
            dim_param: "num_channels"
          }
          dim {
            dim_param: "height"
          }
          dim {
            dim_param: "width"
          }
        }
      }
    }
    }
    output {
    name: "last_hidden_state"
    type {
      tensor_type {
        elem_type: 1
        shape {
          dim {
            dim_param: "batch_size"
          }
          dim {
            dim_param: "Addlast_hidden_state_dim_1"
          }
          dim {
            dim_param: "Addlast_hidden_state_dim_2"
          }
        }
      }
    }
    }
    }
    opset_import {
    version: 14
    }

[INFO] 2024-08-10T12:33:27z thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp 309 Converting Onnx Graph [ERROR] 2024-08-10T12:33:28z thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp 421 Unknown op type: Where [ERROR] 2024-08-10T12:33:28z thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp 421 Unknown op type: ConstantOfShape [ERROR] 2024-08-10T12:33:28z src/nnfusion/util/errors.hpp 169 Failure at /root/nnfusion/thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp:423: Unknown op count: 2 terminate called after throwing an instance of 'nnfusion::errors::CheckError' what(): Failure at /root/nnfusion/thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp:423: Unknown op count: 2 Aborted (core dumped)



**Expected behavior**
Successfully generate CUDA code and run well.

**Additional context**
I use the docker image from your repo, it goes well when I just use the model nnfusion provided in example.