allenai / OLMoE

OLMoE: Open Mixture-of-Experts Language Models
https://arxiv.org/abs/2409.02060
Apache License 2.0
468 stars 37 forks source link

recommended conf #12

Closed raingart closed 1 month ago

Muennighoff commented 1 month ago

For temp, top_k, top_p, I'd just try a bit and use what works best; no recommended setting but don't expect it to be very different from other LMs.

I don't quite follow your example - if you clearly paste the prompt and model response I can take a look