Closed IlyasMoutawwakil closed 3 months ago
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
@echarlaix I added an openvino
variant of bert-with-pooler
in IlyasMoutawwakil/sbert-all-MiniLM-L6-v2-with-pooler
(can move it to optimum if you prefer), and added a custom export test that creates it
What does this PR do?
This PR adds custom tasks modeling through a new class
OVModelForCustomTasks
which allows using models that are exported with arbitrary inputs/outputs.Fixes #611
how to use
For example to use
google/vit-base-patch16-224
model forimage-classification
while also getting all thehidden_states
:main_export
andmodel_kwargs={"output_hidden_states": True}
which is passed to the model at inference time, andcustom_export_configs
which is a dictionary of custom export configs that tells the exporter how to handle inputs/outputs (e.g. what to name them).OVModeForCustomTasks
:from optimum.intel.openvino import OVModelForCustomTasks
image_processor = AutoImageProcessor.from_pretrained(model_id) url = "http://images.cocodataset.org/val2017/000000039769.jpg" image = Image.open(requests.get(url, stream=True).raw) inputs = image_processor(images=image, return_tensors="pt")
ov_model = OVModelForCustomTasks.from_pretrained(output) ov_outputs = ov_model(**inputs) print(ov_outputs.keys()) # dict_keys(['logits', 'hidden_states'])
ov_model.push_to_hub(save_directory=output, repository_id=repo_id)