huggingface / optimum-intel

🤗 Optimum Intel: Accelerate inference with Intel optimization tools

https://huggingface.co/docs/optimum/main/en/intel/index

Apache License 2.0

364 stars 101 forks source link

fix input generator for falcon40b #685

Closed eaidova closed 3 months ago

eaidova commented 3 months ago

What does this PR do?

this is temporal workaround until we can not use https://github.com/huggingface/optimum/pull/1825

falcon dummy input generator is not able to handle falcon-40b model (case with new_decoder_architecture and multi_query)

HuggingFaceDocBuilderDev commented 3 months ago

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

eaidova commented 3 months ago

@echarlaix could you please take a look?