could the checkpoint of bert convert to onnx model? I have a bug that 'BertForPreTraining' object has no attribute 'layers, output'

[Igoslow]

Describe the bug A clear and concise description of what the bug is. could the checkpoint of bert convert to onnx model? I have a bug that 'BertForPreTraining' object has no attribute 'layers, output'

Urgency If there are particular important use cases blocked by this or strict project-related timelines, please share more information and dates. If there are no hard deadlines, please specify none.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): windows
• ONNX Runtime installed from (source or binary): source
• ONNX Runtime version: 1.4.0
• Python version: 3.6.8
• Visual Studio version (if applicable):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version:none
• GPU model and memory: none

To Reproduce

• Describe steps/code to reproduce the behavior.
• Attach the ONNX model to the issue (where applicable) to expedite investigation. AttributeError: 'BertForPreTraining' object has no attribute 'outputs' or 'layers'

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.

[Igoslow]

please help me, thank you

[yufenglee]

@Igoslow , could you add more information of your issue? Are you converting a TF or PyTorch Model? For model conversion, tf2onnx and PyTorch Exporter are better place.

[Igoslow]

Thank you for your reply. I convert a TF model of bert. The my source code are below.

enable_overwrite = False total_runs = 100 max_sequence_length = 512 import os cache_dir = './cache_models' output_dir = './onnx_models' for directory in [cache_dir, output_dir]: if not os.path.exists(directory): os.makedirs(directory) from transformers import BertTokenizer model_name_or_path = "bert-base-cased"

tokenizer = BertTokenizer.from_pretrained(model_name_or_path, do_lower_case=True, cache_dir=cache_dir) from transformers.modeling_bert import BertForPreTraining#,BertForQuestionAnswering,BertModel from transformers.modeling_bert import BertConfig config = BertConfig.from_json_file('./chinese_L-12_H-768_A-12/bert_config.json') model = BertForPreTraining.from_pretrained('./chinese_L-12_H-768_A-12/bert_model.ckpt.index', config=config, from_tf=True)

model._saved_model_inputs_spec = None

import time import keras2onnx output_model_path = os.path.join(outputdir, 'keras{}.onnx'.format(model_name_or_path))

if enable_overwrite or not os.path.exists(output_model_path): start = time.time() onnx_model = keras2onnx.convert_keras(model, 'Bert', target_opset=10) #model.name keras2onnx.save_model(onnx_model, output_model_path) print("Keras2onnx run time = {} s".format(format(time.time() - start, '.2f')))

[mfuntowicz]

Hi @Igoslow,

Morgan from Hugging Face team here 👋.

I suspect you're combining two things incompatible:

1. You load a TensorFlow checkpoint into a PyTorch model

config = BertConfig.from_json_file('./chinese_L-12_H-768_A-12/bert_config.json')
model = BertForPreTraining.from_pretrained('./chinese_L-12_H-768_A-12/bert_model.ckpt.index', config=config, from_tf=True)

=> This will give you a PyTorch module you can leverage to do inference/training with

2. You're trying to export this PyTorch model through keras2onnx but this one only supports TF model.

onnx_model = keras2onnx.convert_keras(model, 'Bert', target_opset=10) #model.name
keras2onnx.save_model(onnx_model, output_model_path)

Possible solution:

If you want to make it works without too much change in your code, you should replace:

from transformers.modeling_bert import TFBertForPreTraining#,BertForQuestionAnswering,BertModel

model = TFBertForPreTraining.from_pretrained('./chinese_L-12_H-768_A-12/bert_model.ckpt.index', config=config)

By doing this, you'll be loading the checkpoint within a TensorFlow object which will be handled correctly by keras2onnx.

[Igoslow]

Thank you very much. I almost understand. I have a try for your advice. But I find other problems as blow.

Traceback (most recent call last): File "E:/roberta-bert-finetune/bert_tensorflow_onnx.py", line 24, in model = TFBertForPreTraining.from_pretrained(model_name_or_path, config=config)#,from_pt=True) File "C:\Users\Learn\Envs\tf2onnx\lib\site-packages\transformers\modeling_tf_utils.py", line 490, in from_pretrained model.load_weights(resolved_archive_file, by_name=True) File "C:\Users\Learn\Envs\tf2onnx\lib\site-packages\tensorflow\python\keras\engine\training.py", line 2185, in load_weights 'Weights may only be loaded based on topology into Models when ' NotImplementedError: Weights may only be loaded based on topology into Models when loading TensorFlow-formatted weights (got by_name=True to load_weights).

[mfuntowicz]

Yes, I think keras2onnx needs some more steps, especially it needs some inputs to trace the model. Can you can something like this:

tokenizer = BertTokenizer.from_pretrained("<path/to/your/tokenizer>")
tokens = tokenizer("Some input data")
model.model.predict(tokens.data)
onnx_model = convert_keras(model.model, model.model.name, target_opset=10)

[Igoslow]

I follow your advice but I get same problem that NotImplementedError: Weights may only be loaded based on topology into Models when loading TensorFlow-formatted weights (got by_name=True to load_weights). My code are as below.

enable_overwrite = False total_runs = 100 max_sequence_length = 512 import os cache_dir = './cache_models' output_dir = './onnx_models' for directory in [cache_dir, output_dir]: if not os.path.exists(directory): os.makedirs(directory) from transformers import (TFBertForPreTraining,BertConfig,BertTokenizer,BertForPreTraining,TFBertModel, TFBertForQuestionAnswering, BertTokenizer) tokenizer = BertTokenizer.from_pretrained('./chinese_L-12_H-768_A-12') model_name_or_path = "orin_model/bert_model.ckpt.index" config = BertConfig.from_json_file('./chinese_L-12_H-768_A-12/bert_config.json') model = TFBertForPreTraining.from_pretrained(model_name_or_path, config=config)#,from_pt=True) model._saved_model_inputs_spec = None model.saved_model_inputs_spec = None import time import keras2onnx output_model_path = os.path.join(output_dir, 'keras{}.onnx'.format(model_name_or_path)) if enable_overwrite or not os.path.exists(output_model_path): start = time.time() onnx_model = keras2onnx.convert_keras(model, model.name, target_opset=10)#model.name) keras2onnx.save_model(onnx_model, output_model_path) print("Keras2onnx run time = {} s".format(format(time.time() - start, '.2f')))

the main of environment are as follow. Keras 2.1.6 keras2onnx 1.6.0 onnx 1.5.0 onnxconverter-common 1.7.0 onnxruntime 1.5.1 onnxruntime-tools 1.4.0 tensorflow 2.3.0 tensorflow-estimator 2.3.0 termcolor 1.1.0 tokenizers 0.8.1rc1 torch 1.6.0+cpu torchvision 0.7.0+cpu