An issue that occurred while optimizing Babelscape/mrebel-large

Describe the issue

The following problem occurred when I optimized Babelscape/mrebel-large： warnings.warn( Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41. Non-default generation parameters: {'max_length': 200, 'num_beams': 5, 'forced_eos_token_id': 2} Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Optimizing model... 2024-03-31 07:08:18.326148500 [W:onnxruntime:, session_state.cc:1166 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf. 2024-03-31 07:08:18.326165852 [W:onnxruntime:, session_state.cc:1168 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments. Failed to run symbolic shape inference. Please file an issue in https://github.com/microsoft/onnxruntime. 2024-03-31 07:08:53.628257884 [W:onnxruntime:, session_state.cc:1166 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf. 2024-03-31 07:08:53.628272405 [W:onnxruntime:, session_state.cc:1168 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments. failed in shape inference <class 'Exception'> Failed to run symbolic shape inference. Please file an issue in https://github.com/microsoft/onnxruntime. failed in shape inference <class 'Exception'> Traceback (most recent call last): File "/xxx/data/app/mrebel-large/mrebelonnx330.py", line 14, in optimizer.optimize(save_dir="mrebelonnx330", optimization_config=optimization_config) File "/home/xxx/anaconda3optium/lib/python3.9/site-packages/optimum/onnxruntime/optimization.py", line 226, in optimize optimizer.save_model_to_file( File "/home/xxxanaconda3optium/lib/python3.9/site-packages/onnxruntime/transformers/onnx_model.py", line 1175, in save_model_to_file self.topological_sort() File "/home/xxx/anaconda3optium/lib/python3.9/site-packages/onnxruntime/transformers/onnx_model.py", line 1118, in topological_sort OnnxModel.graph_topological_sort(self.model.graph, is_deterministic) File "/home/xxx/anaconda3optium/lib/python3.9/site-packages/onnxruntime/transformers/onnx_model.py", line 1107, in graph_topological_sort raise RuntimeError( RuntimeError: Graph is not a DAG: len(sorted_node_set)=912, len(graph.node)=934, failed at node graph_output_cast46

Can you help me see what the problem is？

To reproduce

My code is as follows： from optimum.onnxruntime import ORTModelForSeq2SeqLM from transformers import AutoModelForSeq2SeqLM, AutoTokenizer from optimum.onnxruntime import OptimizationConfig, ORTOptimizer, ORTModelForSeq2SeqLM import torch tokenizer = AutoTokenizer.from_pretrained("Babelscape/mrebel-large") model = ORTModelForSeq2SeqLM.from_pretrained("Babelscape/mrebel-large", from_transformers=True) optimizer = ORTOptimizer.from_pretrained(model) optimization_config = OptimizationConfig( optimization_level=1, enable_transformers_specific_optimizations=True, optimize_for_gpu=True, fp16=True, ) optimizer.optimize(save_dir="mrebelonnx330", optimization_config=optimization_config)

The following software versions are used：

Name: transformers Version: 4.39.2

Name: optimum Version: 1.18.0

Urgency

I want to solve this problem as soon as possible. Please provide me with some constructive suggestions

Platform

Linux

OS Version

No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.3 LTS Release: 22.04 Codename: jammy

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

Name: onnxruntime-gpu Version: 1.17.1

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2022 NVIDIA Corporation Built on Wed_Jun__8_16:49:14_PDT_2022 Cuda compilation tools, release 11.7, V11.7.99 Build cuda_11.7.r11.7/compiler.31442593_0

### Tasks

microsoft / onnxruntime