search

llvm / torch-mlir

The Torch-MLIR project aims to provide first class support from the PyTorch ecosystem to the MLIR ecosystem.
Other
1.34k stars 499 forks source link

How to convert BERT model to MLIR with use_tracing=False #2181

Open shibizhao opened 1 year ago

shibizhao commented 1 year ago

Hi,

I am a beginner of torch-mlir. I can execute the examples/torchscript_stablehlo_backend_tinybert.py successfully and get the MLIR output. I found that the constant matrices are with specific encoded values in the MLIR file. When I set the use_tracing=False, there is some errors in my output. But ResNet18 can generate without tracing.

RuntimeError: 
'Optional[Tensor]' object has no attribute or method 'size'.:
  File "/home/bshi/opt/anaconda3/envs/torch-mlir-dev/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 211
            input_shape = input_ids.size()
        else:
            input_shape = inputs_embeds.size()[:-1]
                          ~~~~~~~~~~~~~~~~~~ <--- HERE

        seq_length = input_shape[1]

I am unfamiliar with the usage of use_tracing. So I would like to ask you for help.

Thank you!

ramiro050 commented 1 year ago

Hi @shibizhao, use_tracing just decides whether to use torch.jit.script or torch.jit.trace to convert the torch.nn.Module being passed into TorchScript. The error you're getting is a TorchScript error caused by torch.jit.script being unable to verify that the types in your program are correct. In other words, it does not seem to be a torch-mlir error.