search

nvtransfer / RULER

This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?
Apache License 2.0
646 stars 43 forks source link

About Mistral-Small-Instruct-2409 #65

Open showgood163 opened 1 month ago

showgood163 commented 1 month ago

Hi there,

Will you test Mistral-Small-Instruct-2409?

22B so not too expensive for a test.

official-elinas commented 1 month ago

I'm trying to, but keep getting ModuleNotFoundError: No module named 'nemo'

Working on trying to fix this.

official-elinas commented 1 month ago

Fixed it but I'm not sure about the engine type / not loading the model anymore.

hsiehjackson commented 1 month ago

You are available to use vLLM or HF to run inference. BTW, I just found they change their claimed context length from 128K to 32K.