Closed VenkateshSrini closed 8 months ago
I took a look at the meta-llama/Llama-2-7b-hf model on Hugging Face and it looks like it's inference API has been disabled. Can you confirm that you have access to the inference API?
@matthewbolanos ,
I tried this end point https://[api-inference.huggingface.co/models/meta-llama/Llama-2-7b-hf](https://api-inference.huggingface.co/models/meta-llama/Llama-2-7b-hf) and I got the below result
{ "id": "meta-llama/Llama-2-7b-hf", "sha": "8cca527612d856d7d32bd94f8103728d614eb852", "pipeline_tag": "text-generation", "library_name": "transformers", "private": false, "gated": "manual", "siblings": [], "safetensors": { "parameters": { "F16": 6738415616, "F32": 2048 } }, "cardData": { "tags": [ "facebook", "meta", "pytorch", "llama", "llama-2" ], "base_model": null } }
I'm ok to use any other LLM model also. My idea is to test the GenAI from hugging face. Can you please suggest any other model that I can use ?
@matthewbolanos , I now tire the same with Microsoft Orca2 13B available on hugging face. The code is as below
private static async Task RunLlamaExampleAsync()
{
Console.WriteLine("\n======== HuggingFace Llama 2 example ========\n");
const string Model = "microsoft/Orca-2-13b";
// HuggingFace local HTTP server endpoint
const string Endpoint = "https://api-inference.huggingface.co/models/microsoft/Orca-2-13b";
Kernel kernel = new KernelBuilder()
.WithLoggerFactory(ConsoleLogger.LoggerFactory)
.WithHuggingFaceTextCompletionService(
model: Model,
endpoint: Endpoint,
apiKey: TestConfiguration.HuggingFace.ApiKey)
.Build();
var questionAnswerFunction = kernel.CreateFunctionFromPrompt("Question: {{$input}}; Answer:");
var result = await kernel.InvokeAsync(questionAnswerFunction, "What is New York?");
Console.WriteLine(result.GetValue<string>());
}
But still I get the old response
fail: Microsoft.SemanticKernel.SKFunctionFactory[0] Prompt function funcc3876625d61544968fcd673648e8dec6 execution failed with error Response status code does not indicate success: 400 (Bad Request). Microsoft.SemanticKernel.Http.HttpOperationException: Response status code does not indicate success: 400 (Bad Request). ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 400 (Bad Request). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at HttpClientExtensions.SendWithSuccessCheckAsync(HttpClient client, HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\InternalUtilities\src\Http\HttpClientExtensions.cs:line 46 --- End of inner exception stack trace --- at HttpClientExtensions.SendWithSuccessCheckAsync(HttpClient client, HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\InternalUtilities\src\Http\HttpClientExtensions.cs:line 50 at HttpClientExtensions.SendWithSuccessCheckAsync(HttpClient client, HttpRequestMessage request, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\InternalUtilities\src\Http\HttpClientExtensions.cs:line 67 at Microsoft.SemanticKernel.Connectors.AI.HuggingFace.TextCompletion.HuggingFaceTextCompletion.ExecuteGetCompletionsAsync(String text, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\Connectors\Connectors.AI.HuggingFace\TextCompletion\HuggingFaceTextCompletion.cs:line 131 at Microsoft.SemanticKernel.Connectors.AI.HuggingFace.TextCompletion.HuggingFaceTextCompletion.GetCompletionsAsync(String text, AIRequestSettings requestSettings, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\Connectors\Connectors.AI.HuggingFace\TextCompletion\HuggingFaceTextCompletion.cs:line 81 at Microsoft.SemanticKernel.KernelFunctionFromPrompt.InvokeCoreAsync(Kernel kernel, ContextVariables variables, AIRequestSettings requestSettings, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\SemanticKernel.Core\Functions\SKFunctionFromPrompt.cs:line 154 fail: funcc3876625d61544968fcd673648e8dec6[0] Function failed. Error: Response status code does not indicate success: 400 (Bad Request). Microsoft.SemanticKernel.Http.HttpOperationException: Response status code does not indicate success: 400 (Bad Request). ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 400 (Bad Request). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at HttpClientExtensions.SendWithSuccessCheckAsync(HttpClient client, HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\InternalUtilities\src\Http\HttpClientExtensions.cs:line 46 --- End of inner exception stack trace --- at HttpClientExtensions.SendWithSuccessCheckAsync(HttpClient client, HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\InternalUtilities\src\Http\HttpClientExtensions.cs:line 50 at HttpClientExtensions.SendWithSuccessCheckAsync(HttpClient client, HttpRequestMessage request, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\InternalUtilities\src\Http\HttpClientExtensions.cs:line 67 at Microsoft.SemanticKernel.Connectors.AI.HuggingFace.TextCompletion.HuggingFaceTextCompletion.ExecuteGetCompletionsAsync(String text, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\Connectors\Connectors.AI.HuggingFace\TextCompletion\HuggingFaceTextCompletion.cs:line 131 at Microsoft.SemanticKernel.Connectors.AI.HuggingFace.TextCompletion.HuggingFaceTextCompletion.GetCompletionsAsync(String text, AIRequestSettings requestSettings, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\Connectors\Connectors.AI.HuggingFace\TextCompletion\HuggingFaceTextCompletion.cs:line 81 at Microsoft.SemanticKernel.KernelFunctionFromPrompt.InvokeCoreAsync(Kernel kernel, ContextVariables variables, AIRequestSettings requestSettings, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\SemanticKernel.Core\Functions\SKFunctionFromPrompt.cs:line 154 at Microsoft.SemanticKernel.KernelFunction.InvokeAsync(Kernel kernel, ContextVariables variables, AIRequestSettings requestSettings, CancellationToken cancellationToken) in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel-main\dotnet\src\SemanticKernel.Abstractions\Functions\KernelFunction.cs:line 113
I wonder if this is because these models are chat completion instead of text generation? We'll take a look at this as we work on getting Llama support for Semantic Kernel.
@matthewbolanos can you please point me to some chat completion models in Hugginface that I can use to test this ?
It looks like I probably mispoke, these were the chat ones I was thinking about: https://huggingface.co/models?pipeline_tag=conversational&sort=trending, but all of the models you're using are text generation, so they should work. I'm going to elevate this as a bug so we can take a look at this sooner rather than later.
@matthewbolanos the issue I'm getting is for microsoft/DialoGPT-medium
Microsoft.SemanticKernel.HttpOperationException HResult=0x80131500 Message=Response status code does not indicate success: 400 (Bad Request). Source=Microsoft.SemanticKernel.Connectors.HuggingFace StackTrace: at Microsoft.SemanticKernel.Http.HttpClientExtensions.
d0.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\src\InternalUtilities\src\Http\HttpClientExtensions.cs:line 51 at Microsoft.SemanticKernel.Http.HttpClientExtensions. d 1.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\src\InternalUtilities\src\Http\HttpClientExtensions.cs:line 68 at Microsoft.SemanticKernel.Connectors.HuggingFace.HuggingFaceTextGenerationService.d12.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\src\Connectors\Connectors.HuggingFace\TextGeneration\HuggingFaceTextGenerationService.cs:line 112 at Microsoft.SemanticKernel.TextGeneration.TextGenerationExtensions. d 0.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\src\SemanticKernel.Abstractions\AI\TextGeneration\TextGenerationExtensions.cs:line 33 at Microsoft.SemanticKernel.TextGeneration.TextGenerationExtensions.d1.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\src\SemanticKernel.Abstractions\AI\TextGeneration\TextGenerationExtensions.cs:line 60 at Microsoft.SemanticKernel.KernelFunctionFromPrompt. d 3.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\src\SemanticKernel.Core\Functions\KernelFunctionFromPrompt.cs:line 140 at System.Threading.Tasks.ValueTask`1.get_Result() in System.Threading.Tasks\ValueTask.cs:line 484 at Microsoft.SemanticKernel.KernelFunction.d18.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\src\SemanticKernel.Abstractions\Functions\KernelFunction.cs:line 142 at Example20_HuggingFace. d1.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\samples\KernelSyntaxExamples\Example20_HuggingFace.cs:line 32 at Example20_HuggingFace. 5.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\samples\KernelSyntaxExamples\Program.cs:line 107 at Program.d 0.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\samples\KernelSyntaxExamples\Example20_HuggingFace.cs:line 12 at Program.d d2.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\samples\KernelSyntaxExamples\Program.cs:line 65 at Program. d 1.MoveNext() in C:\CarryCase\Venkatesh\ML\SemanticKernel\semantic-kernel\dotnet\samples\KernelSyntaxExamples\Program.cs:line 36 at Program.(String[] args) This exception was originally thrown at this call stack: System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() Microsoft.SemanticKernel.Http.HttpClientExtensions.SendWithSuccessCheckAsync(System.Net.Http.HttpClient, System.Net.Http.HttpRequestMessage, System.Net.Http.HttpCompletionOption, System.Threading.CancellationToken) in HttpClientExtensions.cs
Inner Exception 1: HttpRequestException: Response status code does not indicate success: 400 (Bad Request).
Did anyone solve this? I cannot access any hugging face models from vscode semantic-kernel extension
All .Net issues prior to 1-Dec-2023 are being closed. Please re-open, if this issue is still relevant to the .Net Semantic Kernel 1.x release. In the future all issues that are inactive for more than 90 days will be labelled as 'stale' and closed 14 days later.
The issue is still not resolved
Agreed, having the same problem. @VenkateshSrini you may want to reopen this issue since they have closed it.
Hello, I am actually having the same issue, did anyone resolve it or manage to find what's wrong?
Describe the bug I'm trying to run hugging face semantic kernel example. I have provided the configuration as below
{ "HuggingFace": { "ApiKey": "xxxxxxxxxxxxx", "ModelId": "meta-llama/Llama-2-7b-hf" } }
I'm not getting desired result To Reproduce Steps to reproduce the behavior: 1) Get the Access token from settings option 2) Provide it code via secrets.jsonExpected behavior I should get response to the chat message that I sent
Exception details for Inference API
Exception Details for Meta LLM
Platform
Additional context Add any other context about the problem here.