DummyTextInputGenerator doesn't take into account type_vocab_size model config parameter

mrapplexz commented 6 months ago

System Info

Optimum 1.18.0, Transformers 4.38.1, PyTorch 2.2.1+cu121
CUDA Version: 12.4
Platform: Arch Linux x86_64
Python: 3.10.11

Who can help?

@michaelbenayoun

Information

[ ] The official example scripts
[ ] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[X] My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)

optimum-cli export onnx --model Tochka-AI/ruRoPEBert-e5-base-2k --optimize O1 --task feature-extraction ruRoPEBert-e5-base-2k-onnx --trust-remote-code

Framework not specified. Using pt to export the model.
Using the export variant default. Available variants are:
    - default: The default ONNX variant.
Using framework PyTorch: 2.2.1+cu121
Traceback (most recent call last):
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/bin/optimum-cli", line 8, in <module>
    sys.exit(main())
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/optimum/commands/optimum_cli.py", line 163, in main
    service.run()
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/optimum/commands/export/onnx.py", line 261, in run
    main_export(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/optimum/exporters/onnx/__main__.py", line 351, in main_export
    onnx_export_from_model(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/optimum/exporters/onnx/convert.py", line 1157, in onnx_export_from_model
    _, onnx_outputs = export_models(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/optimum/exporters/onnx/convert.py", line 768, in export_models
    export(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/optimum/exporters/onnx/convert.py", line 873, in export
    export_output = export_pytorch(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/optimum/exporters/onnx/convert.py", line 577, in export_pytorch
    onnx_export(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/onnx/utils.py", line 516, in export
    _export(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/onnx/utils.py", line 1613, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/onnx/utils.py", line 1135, in _model_to_graph
    graph, params, torch_out, module = _create_jit_graph(model, args)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/onnx/utils.py", line 1011, in _create_jit_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/onnx/utils.py", line 915, in _trace_and_get_graph_from_model
    trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/jit/_trace.py", line 1296, in _get_trace_graph
    outs = ONNXTracedModule(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/jit/_trace.py", line 138, in forward
    graph, out = torch._C._create_graph_by_tracing(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/jit/_trace.py", line 129, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/optimum/exporters/onnx/model_patcher.py", line 149, in patched_forward
    outputs = self.orig_forward(*args, **kwargs)
  File "/home/me/.cache/huggingface/modules/transformers_modules/Tochka-AI/ruRoPEBert-e5-base-2k/2cb99471f875086cb684708d195d6e7db13f6644/modeling_rope_bert.py", line 853, in forward
    embedding_output = self.embeddings(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/me/.cache/huggingface/modules/transformers_modules/Tochka-AI/ruRoPEBert-e5-base-2k/2cb99471f875086cb684708d195d6e7db13f6644/modeling_rope_bert.py", line 141, in forward
    token_type_embeddings = self.token_type_embeddings(token_type_ids)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 163, in forward
    return F.embedding(
  File "/home/me/.cache/pypoetry/virtualenvs/tesla-9HRTWqlD-py3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 2237, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self

Expected behavior

The model will be successfully exported.

mrapplexz commented 6 months ago

Hello! After some research, I figured out that the problem is related to Optimum's dummy input generators. The DummyTextInputGenerator always use 2 as a max value for token type IDs, not taking into account a value specified in type_vocab_size model config field. Therefore, indexing error occurs while tracing a graph of a model that has "type_vocab_size": 1.

gulldan commented 3 days ago

same problem, any update here?

huggingface / optimum