Mixtral sliding window - Githubissues

mistralai / mistral-inference

Official inference library for Mistral models

https://mistral.ai/

Apache License 2.0

9.58k stars 845 forks source link

Open tuyaao opened 8 months ago

tuyaao commented 8 months ago

What is the sliding window size for Mixtral training, Is it 32k or 8k