search

mlcommons / modelbench

Run safety benchmarks against AI models and view detailed reports showing how well they performed.
https://mlcommons.org/ai-safety/
Apache License 2.0
62 stars 11 forks source link

Stand up mistral SUT adapter for ministral 8b instruct #688

Open rogthefrog opened 1 week ago

rogthefrog commented 1 week ago

To test: send too many requests, see how it fails, and add retry logic.

Look at the together SUT adapter for existing retry logic.

rogthefrog commented 3 days ago

Which service do we want to use for ministral 8b instruct? @bollacker

rogthefrog commented 3 days ago

We're using ministral-8b-latest on mistral AI (not instruct).