ONNX export XLM-RoBERTa

Mrpatekful commented 4 years ago

I have tried exporting the xlmr.base model from torch.hub to onnx, but I receive an exception regarding ATen.

  %output : Float(1, 5, 768) = onnx::Transpose[perm=[1, 0, 2]](%1188), scope: Exported/RobertaModel[model]/RobertaEncoder[decoder] # /home/ppatrik/.cache/torch/hub/pytorch_fairseq_master/fairseq/models/roberta/model.py:367:0
  return (%output)

Warning: ATen was a removed experimental ops. In the future, we may directly reject this operator. Please update your model as soon as possible.
Traceback (most recent call last):
  File "/home/ppatrik/.conda/envs/condaenv/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/ppatrik/.conda/envs/condaenv/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/mnt/permanent/home/ppatrik/entity-recognition/src/export.py", line 168, in <module>
    main()
  File "/mnt/permanent/home/ppatrik/entity-recognition/src/export.py", line 152, in main
    output_path)
  File "/home/ppatrik/.conda/envs/condaenv/lib/python3.7/site-packages/onnxruntime/capi/session.py", line 25, in __init__
    self._load_model(providers)
  File "/home/ppatrik/.conda/envs/condaenv/lib/python3.7/site-packages/onnxruntime/capi/session.py", line 43, in _load_model
    self._sess.load_model(providers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: ATen is not a registered function/op

oysz2016 commented 4 years ago

I met the same problem, have you solved it?

Mrpatekful commented 4 years ago

Unfortunately I could not. If a recall correctly an error originated from the cumsum operation (which is included in onnx 1.5 with opset 11) in https://github.com/pytorch/fairseq/blob/9f25ffb02ab54a443ba181f732d5dcc00db7bea8/fairseq/utils.py#L192 . However I might have fixed that and one more error persisted so I gave up and implemented my own fairseq model (Not sure if I fixed it and still got an error or just didn't bother and went straight to implementing my own). https://github.com/Mrpatekful/xlmr-finetuning I tested the outputs of the two version and they are identical with atol=1e-5, but mine could be exported without and error with onnx>=1.5 .

facebookresearch / fairseq

ONNX export XLM-RoBERTa #1644