huggingface / optimum-intel

🤗 Optimum Intel: Accelerate inference with Intel optimization tools

https://huggingface.co/docs/optimum/main/en/intel/index

Apache License 2.0

415 stars 112 forks source link

allow to use SDPA in clip models #941

Closed eaidova closed 1 month ago

eaidova commented 1 month ago

What does this PR do?

Fixes # (issue)

Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

eaidova commented 1 month ago

Thanks @eaidova ! Opened huggingface/optimum#2066 to remove CLIPModelPatcher direclty in optimum (shouldn't be needed for onnx), would that works for you ?

@echarlaix thanks, but are you sure that it is not need? as I understand @IlyasMoutawwakil met integration issue for onnx export that is why it was added

echarlaix commented 1 month ago

@echarlaix thanks, but are you sure that it is not need? as I understand @IlyasMoutawwakil met integration issue for onnx export that is why it was added

Actually might be needed for diffusers model, let's merge this PR for now and we can always revert changes if updated in optimum

IlyasMoutawwakil commented 1 month ago

the specific onnx error was https://github.com/pytorch/pytorch/issues/135615 and it was fixed in latest pytorch (not sure if stable or nightly)