onnx / onnx-mlir

Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure
Apache License 2.0
744 stars 315 forks source link

Example ONNX model (mnist.onnx) fails to compile in FP16 precision #1573

Open gcunhase opened 2 years ago

gcunhase commented 2 years ago

How to reproduce

  1. Download mnist.onnx model at https://github.com/onnx/onnx-mlir/blob/main/docs/mnist_example/mnist.onnx

  2. Install ONNX-MLIR:

    
    # Pull the latest docker image:
    $ docker pull onnxmlirczar/onnx-mlir

Test that it's working

$ git clone https://github.com/onnx/onnx-mlir.git $ cd onnx-mlir/docker $ python onnx-mlir.py --help


3. Install additional requirements:

$ pip install onnxmltools


4. Convert FP32 to FP16 model via a simple python script:
```python
import onnxmltools
from onnxmltools.utils.float16_converter import convert_float_to_float16

def convert_onnx_fp32_to_fp16(fp32_model_path, fp16_model_path=None):
    if not fp16_model_path:
        fp16_model_path = fp32_model_path.replace(".onnx", "_fp16.onnx")
    model = onnxmltools.utils.load_model(fp32_model_path)
    model_fp16 = convert_float_to_float16(model)
    onnxmltools.utils.save_model(model_fp16, file_path=fp16_model_path)

convert_onnx_fp32_to_fp16("./mnist.onnx", "./mnist_fp16.onnx")
  1. Compile .onnx to .so file:
    $ python onnx-mlir.py --O3 --EmitLib ./mnist_fp16.onnx

Observed behavior

$ python onnx-mlir.py --O3 --EmitLib ./models/mnist_fp16.onnx
Failed to import ONNX TensorProto due to unsupported data types.
UNREACHABLE executed at /onnx-mlir/src/Builder/FrontendDialectHelper.cpp:241!
Unknown error 139

Expected behavior

Generate a .so file for the FP16 model.

System info

Ubuntu 18.04, Python 3.8 ONNX-MLIR:

$ python onnx-mlir.py --version
onnx-mlir version 0.3.0, onnx version 1.11.0 (https://github.com/onnx/onnx-mlir.git 9c6b1ea393685eda3f78ea064fcb358b5091d778)
  LLVM version 15.0.0git (https://github.com/llvm/llvm-project.git f2b94bd7eaa83d853dc7568fac87b1f8bf4ddec6)
  Optimized build with assertions.
  Default target: x86_64-unknown-linux-gnu
  Host CPU: cascadelake
AlexandreEichenberger commented 2 years ago

@gcunhase I don't think we (IBM) have machines executing fp16 on CPU, so we have not exercised this path at this time. We have an accelerator (NNPA) that uses fp16 but we only convert to/from on CPU for exclusive usage on the accelerator. We will probably extend some processing on CPU but we have done so yet at this time.

We would love to have increased type coverage. If you want this support, I believe you will have to contribute support, and we are happy to assist you in this endeavor as well as we can. Testing could be done initially via the lit-tests to ensure that it does what is expected.