-
### Summary
# Motivation
The WasmEdge Runtime is poised to offer robust inference support for AI models and Large
Language Models (LLMs) such as llama3 and phi-3-mini. Recognizing the critical …
-
Very interesting application of LLMs in the domain of Causal Inference!
I would like to replicate your results using your code, (not using GPT4 as LLM, but one or more of the open LLMs from GROQ, s…
-
`@prompt(
"Say hello",
model=LitellmChatModel(
model = "gpt-4o",
api_base = "",
custom_llm_provider="azureopen",
extra_header= {
"Api-Key": …
-
**Describe the bug**
Context chat queries are failing today with a 500 internal server error in the context_chat_backend docker container.
Could this be because the Docker container was restarted …
-
I'd like to use VTune to profiel IPEX-llm application focusing on GPU. e.g. the performance/all-in-one benchmark to get a full picture of bottleneck.
My questions are:
1. General guide to use VTun…
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x ] I am running the latest code. Development is very rapid so there are no tagged versions as o…
-
**Describe the bug**
As a first timer, I tried the openai instrumentation, and sent a trace to a local collector (using ollama as the backend). Then I compared the output with [llm semantics defined …
-
### Feature request
This is a Bert based model however when trying to run, the message says model not supported. https://huggingface.co/meta-llama/Prompt-Guard-86M/tree/main
### Motivation
LLM-pow…
-
**Summary**
This enhancement aims to separate the usage of local large language models (LLMs) from LLMs accessed through APIs, and reduce the dependency on installed packages like NVIDIA and PyTorch.…
-
Based on the Semantic Conventions 1.27.0. See [docs](https://opentelemetry.io/docs/specs/semconv/gen-ai/).
This issue is to keep track of the work I've been doing for the Spring AI initial adoption o…