Converting Phi-3.5-vision-instruct to onnx format fails

[MaxAkbar]

Please provide us with the following information:

This issue is for Phi-3.5-vision-instruct: (mark with an x)

- [ X ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Follow instructions for converting the Phi-3.5-vision-instruct to onnx format.

Any log messages given by the failure

(phi35) C:\Users**\source\repos\models>python build.py -i .\microsoft\Phi-3.5-vision-instruct -o .\vision-cpu-fp32 -p fp32 -e cpu C:\Users**\anaconda3\envs\phi35\Lib\site-packages\transformers\models\auto\image_processing_auto.py:513: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use slow_image_processor_class, or fast_image_processor_class instead warnings.warn( Traceback (most recent call last): File "C:\Users**\source\repos\models\build.py", line 241, in model = AutoModelForCausalLM.from_pretrained(args.input, trust_remote_code=True, torch_dtype=args.precision).to(args.execution_provider) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users**\anaconda3\envs\phi35\Lib\site-packages\transformers\models\auto\auto_factory.py", line 551, in from_pretrained model_class = get_class_from_dynamic_module( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users**\anaconda3\envs\phi35\Lib\site-packages\transformers\dynamic_module_utils.py", line 514, in get_class_from_dynamic_module return get_class_in_module(class_name, final_module) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users**\anaconda3\envs\phi35\Lib\site-packages\transformers\dynamic_module_utils.py", line 212, in get_class_in_module module_spec.loader.exec_module(module) File "", line 940, in exec_module File "", line 241, in _call_with_frames_removed File "C:\Users****.cache\huggingface\modules\transformers_modules\Phi-3.5-vision-instruct\modeling_phi3_v.py", line 50, in from .image_embedding_phi3_v_for_onnx import Phi3ImageEmbedding # uncomment when running ONNX export ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'transformers_modules.Phi-3'

Expected/desired behavior

Convert the model to onnx format.

OS and Version?

Microsoft Windows 11 Pro (10.0.26120 Build 26120)

azd version?

Not using azd

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

[kinfey]

can you give me your transformer library version

[MaxAkbar]

Here you go:

Name: transformers
Version: 4.44.2
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: C:\Users\****\anaconda3\envs\phi35\Lib\site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by:

[kinfey]

can you confirm you have added https://huggingface.co/lokinfey/Phi-3.5-vision-instruct-onnx-cpu/blob/main/onnx/image_embedding_phi3_v_for_onnx.py in convert folder @MaxAkbar

[MaxAkbar]

Thank you I did download all of the files but I will retest and try it again tomorrow.

[MaxAkbar]

After a few tries it seems to have worked will test out the onnx model but we can lose this for now. Thank you.