search

aksoftware98 / phi3-dotnet8

.NET 8.0 C# Console app with local host API that hosts the Microsoft Phi-3 mini model locally.
9 stars 3 forks source link

Modify genai_config.json info #1

Open ScottyBlazor opened 5 months ago

ScottyBlazor commented 5 months ago

Great sample thanks. I found the AI answers the question and then starts rambling on about unrelated topics. after checking the doc on Phi2 here https://onnxruntime.ai/docs/genai/reference/config.html

I found it is better to replace the JSON property "eos_token_id": [ 32000, 32001, 32007 ], with "eos_token_id": 32000, rather than removing it altogether it errors when using the array as initial set, but with this single value set it stops it jumping to a different topic once it answers the question.

I still occasionally get it repeating it answer over and over and haven't current found a way to stop that.

aksoftware98 commented 5 months ago

Thank you for the great feedback and useful info.

Actually, as an update for the content I will release it today. Microsoft related the version 0.2 rc of the packages which supports phi3 out-of-the-box without any modifications. Make sure to upgrade the GenAI packages in the sample and it will work pretty well without any changes.

I will keep the issue open until I apply the required modifications to the documentation and the sample.

Thanks, Ahmad

ScottyBlazor commented 5 months ago

Thanks Ahmad we have updated our test code and are now running Microsoft.ML.OnnxRuntimeGenAI 0.2.0-rc6 it runs much better than the older version. thanks for helping us figure out how to get started with using Phi-3-mini in C#