-
目前计划添加的数据:
- [Uncheatable Eval](https://github.com/Jellyfish042/uncheatable_eval):使用最新的动态数据测试 LLM 性能,包含 RWKV
- [RULER_RWKV](https://github.com/Ojiyumm/RULER_RWKV):RWKV 模型的 [RULER](https://arxiv.org/…
-
Not sure if this feature belongs to this library or would it require a complete separate library. I am proposing the creation of a library where llm benchmarks can be ran. For example, evaluating a mo…
-
### System Info
PyTorch: 2.3
Cuda: 12.1
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### 🐛 Describe the bug
I got error when i ran the command generated from …
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
When evaluating a RAG retrieval service using the llama-index evaluation method, I encou…
-
Thank you very much for doing such great open-source work!
i try:
CUDA_VISIBLE_DEVICES=X bash scripts/evaluate.sh PATH_OR_NAME_TO_BASE_MODEL PATH_TO_SAVE_TUNE_MODEL PATH_TO_PRUNE_MODEL EPOCHS_YOU…
-
I tried the documentation:
llm:
api_type: 'openrouter'
base_url: 'https://openrouter.ai/api/v1'
api_key: 'sk...'
model: meta-llama/llama-3-70b-instruct:nitro
Then I got this issu…
-
https://llmc.nii.ac.jp/topics/post-707/
-
Hello, thank you for providing this excellent model and repository. I encountered an issue while conducting my experiments with your codebase, and I’d appreciate your insights.
In my experiments, I…
-
### Is there an existing issue for the same bug?
- [X] I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
- [X] I have checked the existing iss…