Support for Florence 2 model

BrainSlugs83 commented 2 months ago

Feature request

When trying to export Florence 2, it fails with a bizarre error message that leads me to believe it's not supported.

D:\Redacted\>optimum-cli export onnx --model microsoft/Florence-2-large --trust-remote-code --framework pt flo2
D:\miniconda3\envs\onnx-export\Lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\miniconda3\envs\onnx-export\Scripts\optimum-cli.exe\__main__.py", line 7, in <module>
  File "D:\miniconda3\envs\onnx-export\Lib\site-packages\optimum\commands\optimum_cli.py", line 163, in main
    service.run()
  File "D:\miniconda3\envs\onnx-export\Lib\site-packages\optimum\commands\export\onnx.py", line 265, in run
    main_export(
  File "D:\miniconda3\envs\onnx-export\Lib\site-packages\optimum\exporters\onnx\__main__.py", line 280, in main_export
    model = TasksManager.get_model_from_task(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\miniconda3\envs\onnx-export\Lib\site-packages\optimum\exporters\tasks.py", line 1950, in get_model_from_task
    model = model_class.from_pretrained(model_name_or_path, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\miniconda3\envs\onnx-export\Lib\site-packages\transformers\models\auto\auto_factory.py", line 566, in from_pretrained
    raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers_modules.microsoft.Florence-2-large.ef29c9b007f906bd278c39bc12ae620398d88c88.configuration_florence2.Florence2Config'> for this kind of AutoModel: AutoModelForVision2Seq.
Model type should be one of BlipConfig, Blip2Config, GitConfig, Idefics2Config, InstructBlipConfig, Kosmos2Config, LlavaConfig, LlavaNextConfig, PaliGemmaConfig, Pix2StructConfig, VideoLlavaConfig, VipLlavaConfig, VisionEncoderDecoderConfig.

Motivation

I want to export an onnx model of Florence 2. I kind of thought that's the type of thing this tool is used for, yeah? Take a non-onnx repo from hugging face, and export an onnx model.