search

OpenGenerativeAI / llm-colosseum

Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM
https://huggingface.co/spaces/junior-labs/llm-colosseum
MIT License
1.34k stars 160 forks source link

[LLM] Add solar model #26

Closed Tokkiu closed 8 months ago

Tokkiu commented 8 months ago

Add our solar client to this amazing repo. You may refer to more details at https://github.com/run-llama/llama_index/tree/main/llama-index-integrations/llms/llama-index-llms-solar

oulianov commented 8 months ago

Hey ! Thank you for your contribution. I don't know about this model and couldn't find how to get an API key. :/ You sent a link to the Llama index repo. Do you have a paper or a blog post to share as well ?

Tokkiu commented 8 months ago

Hey ! Thank you for your contribution. I don't know about this model and couldn't find how to get an API key. :/ You sent a link to the Llama index repo. Do you have a paper or a blog post to share as well ?

I think this doc will help. https://developers.upstage.ai/product-guides/solar

hunkim commented 8 months ago

@oulianov , this is the enhanced API version of https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0, based on https://arxiv.org/abs/2312.15166. Developers can get their API key from https://console.upstage.ai/. 😊 Let us know if you need more information.

oulianov commented 8 months ago

OK, super clear! Thank you for the details. Adding this model is a great idea!

However, due to refactoring and separation of concerns, the API reference endpoints are fetched in another module. I've added Solar to this other module : https://github.com/phospho-app/phospho/pull/162

For now I'm not sure how to exactly manage adding more LLMs or contributions to "expand the colosseum", but we'll eventually figure this out.

Does this work for you for now?

oulianov commented 8 months ago

Sounds like everything is ok