search

google-ai-edge / ai-edge-torch

Supporting PyTorch models with the Google AI Edge TFLite runtime.
Apache License 2.0
378 stars 51 forks source link

Set ROPE base theta in AttentionConfig. #266

Closed copybara-service[bot] closed 2 months ago

copybara-service[bot] commented 2 months ago

Set ROPE base theta in AttentionConfig.

It's for Qwen model which use the same architecture of Llama, except base theta value.