This repo contains a torchsharp implementation for phi model.
To run the Phi model on your local machine, the following prerequisites are required:
To get Phi-2 model weight, run the following command to download model weight from huggingface. Be sure to have git lfs installed.
git clone https://huggingface.co/microsoft/phi-2
[!Note] To run Phi-2 model on GPU, it's recommended to use a machine with at least 8GB of GPU memory.
[!Note] Loading other Phi model should be similar but I haven't test them yet. Please create an issue if you have trouble loading other Phi models.
Clone this repo and replace the phi2Folder
folder with where you download huggingface model weight in Program.cs
Then run the following command to start the model:
dotnet run
var chatPrompt = @"Alice: I don't know why, I'm struggling to maintain focus while studying. Any suggestions?
Bob: Well, have you tried creating a study schedule and sticking to it?
Alice: Yes, I have, but it doesn't seem to help much.
Bob: Hmm, maybe you should try studying in a quiet environment, like the library.
Alice:";
var chatOutput = phi2.Generate(tokenizer, chatPrompt, maxLen: 256, temperature: 0.3f, stopSequences: [ "Bob:"]);
Console.WriteLine(chatOutput);
Alice: I don't know why, I'm struggling to maintain focus while studying. Any suggestions?
Bob: Well, have you tried creating a study schedule and sticking to it?
Alice: Yes, I have, but it doesn't seem to help much.
Bob: Hmm, maybe you should try studying in a quiet environment, like the library.
Alice: That's a good idea. I'll give it a try. Thanks, Bob!
var prompt = @"Instruction: A skier slides down a frictionless slope of height 40m and length 80m, what's the skier's speed at the bottom?
Output:";
var output = phi2.Generate(tokenizer, prompt, maxLen: 256, temperature: 0.1f);
Console.WriteLine(output);
Instruction: A skier slides down a frictionless slope of height 40m and length 80m, what's the skier's speed at the bottom?!
Output: The skier's speed at the bottom of the slope can be calculated using the conservation of mechanical energy. At the top of the slope, the skier has gravitational potential energy (PE) and no kinetic energy (KE). At the bottom of the slope, the skier has only kinetic energy (KE) and no potential energy (PE). Therefore, the initial potential energy is equal to the final kinetic energy:
PE(top) = KE(bottom)
mgh = (1/2)mv^2
where m is the mass of the skier, g is the acceleration due to gravity, h is the height of the slope, and v is the speed of the skier at the bottom.
Solving for v, we get:
v = sqrt(2gh)
Plugging in the given values, we get:
v = sqrt(2 * 9.8 * 40)
v = 28.28 m/s
So, the skier's speed at the bottom of the slope is 28.28 m/s
var codePrompt = @"Complete the following code
```python
def print_prime(n):
# print all prime numbers less than n";
var codeOutput = phi2.Generate(tokenizer, codePrompt, maxLen: 256, temperature: 0f, stopSequences: [ "```"]);
Console.WriteLine(codeOutput);
Complete the following code
```python
def print_prime(n):
# print all prime numbers less than n
for i in range(2, n):
for j in range(2, i):
if i % j == 0:
break
else:
print(i)
print_prime(10)
Phi model is a suite of small language models developed by Microsoft Research. Until January 2024, there are three models available as follows:
[!Warning] All the Phi-series models does not fined tuned with RLHF, which means the model might generate offensive content. Please use with caution.