huggingface / optimum-intel

🤗 Optimum Intel: Accelerate inference with Intel optimization tools

https://huggingface.co/docs/optimum/main/en/intel/index

Apache License 2.0

355 stars 99 forks source link

[OV Optimum] Keep ShapeOf on Parameter / ReadValue in case of added beam_idx -> Gather #761

Closed jane-intel closed 3 weeks ago

jane-intel commented 3 weeks ago

What does this PR do?

In the case of inserted beam_idx -> Gather, we keep ShapeOf operations on the original Parameter operations (ReadValue operations after the model becomes stateful). This helps avoid special handling of such ShapeOf operations in runtime (CVS-143648).

Before submitting

[N/A] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[N/A] Did you make sure to update the documentation with your changes?
[N/A] Did you write any new necessary tests?

jane-intel commented 3 weeks ago

We've chosen a different approach.