-
1. llmware-ai/[llmware](https://github.com/llmware-ai/llmware): Unified framework for building enterprise RAG pipelines with small, specialized models (github.com)
2. https:[/](https://github.com/ll…
-
We have a new project involving multilingual retrieval and reproduction and we are looking for 2 URA students to work together.
Feel free to reach out on Slack or email us at nandant@gmail.com, xzh…
-
Today, the [Python Evaluation building block](https://aka.ms/azai/eval) can be used against a .NET backend that uses the Chat Protocol (Azure Search supports this). However, we know from customer feed…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
I am trying to use the built-in capabilities of llamaindex to evaluate the correctness o…
-
Great work and thanks for the codebase!
I want to know the exact detailed of LoRA fine-tuning as mentioned in Table 6 of the main paper.
Also if you could point-out to the bash script to reproduce…
-
Example of command:
```python benchmark_throughput.py --model gpt2 --input-len 256 --output-len 256```
Output:
```Namespace(backend='vllm', dataset=None, input_len=256, output_len=256, model='gpt…
-
**Is your feature request related to a problem? Please describe.**
As of now, Haystack's evaluators which extend LLMEvaluator only support OpenAI. I would like for support through llama.cpp to be add…
-
Hello,
Thank you for sharing such an excellent dataset.
The evaluation of Korean models is always a challenging topic, and the information you have provided is greatly beneficial for the develop…
-
Thank you again for your excellent work. I have trained a model mT0 using my own dataset, and it performs well. Now, I am attempting to train bloomz model, but I'm encountering an issue where the trai…
-
I would like to re-open issue #104
There's an overuse of exact matches in the eval harness. For example, consider task 649:
```
"intent": "Post in history subreddit about what could diffusion…