Swap default model from mistral to granite-7b-lab

containers / ai-lab-recipes

Examples for building and running LLM services and applications locally with Podman

Apache License 2.0

104 stars 106 forks source link

Swap default model from mistral to granite-7b-lab #311

Closed Gregory-Pereira closed 5 months ago

Gregory-Pereira commented 5 months ago

Summit is coming, and with it will be information tying the instructlab models to our organization. We should be using our own tools and assets, so we should swap over to using our default model of instructlab/granite-7b-lab. This should be referenced in all our documentation as well as our actual MS / recipes.

Still have yet to hear thoughts about this from @rhatdan @sallyom @lmilbaum

MichaelClifford commented 5 months ago

I have run both instructlab/granite-7b-lab and instructlab/merlinte-7b-lab with the main branch build of llamacpp_python server using the `MODEL_CHAT_FORMAT=openchat" for both the chat application and summarizer (the only two recipes on AI Lab where we'd make this change.

I can confirm that both recipes work with both models.

MichaelClifford commented 5 months ago

issue to get these models included in the AI Lab model catalog https://github.com/containers/podman-desktop-extension-ai-lab/issues/979

rhatdan commented 5 months ago

Can you open a PR to change the default to use granite-7b-lab?

Gregory-Pereira commented 5 months ago

Closing as this is dealt with in #350

devop-mmcgrath commented 5 months ago

Can anyone confirm this also means Granite will show up in the list of models? I saw a demo earlier today and noticed it was missing from Models -> all list.

MichaelClifford commented 5 months ago

@devop-mmcgrath Yeah, we have granite in our model list, and its also propagated into the AI Lab UI.

https://github.com/containers/ai-lab-recipes/tree/main/models