-
add a section about testing llms, this is crucial
-
### Affected component
llms/ShuttleAIToolModel.py
### Motivation
Our testing indicates changes in the ShuttleAIModel, which have surfaced JSON-related errors:
--
FAILED tests/llms/ShuttleAIModel_t…
-
**Is the feature request related to a problem?**
Yes. Currently, Gorilla has a limited set of APIs that it can access. This limits the functionality and potential use cases for LLMs using Gorilla. Ad…
-
### 🚀 The feature, motivation and pitch
I believe, this is what ollama's one huge advantage. This can also encourage devs to go test llm which they can run on their machine's capabilities. Like me,…
-
My use cases may be non-traditional but sharing as there may be others.
I often am using the eval outputs for further data analysis (e.g., in a spreadsheet). I'd prefer if the promptfoo generated t…
-
We need to enable our users to prevent having regressions on their prompt without noticing it clearly. In order to achieve this, we must implement a way for our users to run a test suite on their own …
-
Hello, FlexFlow team!
Thank you for your outstanding work! I am attempting to reproduce the experimental results from the paper "SpecInfer: Accelerating Generative Large Language Model Serving with…
-
## Current Code
Used the [pipecat example code here](https://github.com/pipecat-ai/pipecat/blob/main/examples/foundational/15-switch-voices.py) to define the context and pass it to OpenAILLMContext.
…
-
- [ ] Using pdfs used for QA creation, creating KG
- [ ] Testing on those pdfs
- [ ] Prompt tuning for entities linked below
- [ ] Testing using LLM eval.
[Microsoft graph rag repo link ](http…
-
Hi @Akintunde102 @AyoOdumark ,
if you're trying to test other LLMs (codellama, wizardcoder, etc.) with GPT-Engineer, we just open-sourced a 1-click proxy to translate openai calls to huggingface, …