Support for Microsoft Phi-2 model

[fishfree]

https://huggingface.co/microsoft/phi-2

[localai-bot]

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

I'm not sure what the issue is with the link you provided, but it seems to be related to a model called "phi-2" from the Microsoft Cognitive Services API. If you could provide more context or information about the issue, I can try to help further.

Sources:

• https://github.com/go-skynet/LocalAI/tree/master/README.md
• https://github.com/go-skynet/LocalAI/tree/master/docs/content/features/image-generation.md
• https://localai.io/features/image-generation/index.html
• https://github.com/go-skynet/LocalAI/tree/master/docs/content/_index.en.md

[mudler]

this should actually just work with current LocalAI: https://huggingface.co/TheBloke/phi-2-GGUF

[mudler]

This works here with current master and this YAML config (note GPU settings):

name: phi-2
context_size: 2048
f16: true
threads: 11
gpu_layers: 90
mmap: true
parameters:
  model: huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
  temperature: 0.2
  top_k: 40
  top_p: 0.95
template:
  chat: &template |
    Instruct: {{.Input}}
    Output:
  completion: *template

[mudler]

keeping it open just to have the example added here https://github.com/mudler/LocalAI/tree/master/examples/configurations