-
README 中的演示效果看起来不错，是否可以同时加入一些常见的评测指标（例如 Qwen 给出的：https://huggingface.co/Qwen/Qwen2-72B#qwen2-72b-performance），以和现有的 LLM 进行对比。
目前文档中只有 C-Eval，四选一总体 25% 左右的准确率似乎并不能说明模型学会了知识。
不过作为参数量远小于常规（甚至小于 BER…
-
Hi, I'm from NUS-NCS Cybersecurity Laboratory in SG.
I am interested in using the S-Eval dataset in our LLM risk evaluations. From the README.md file, there's a breakdown of how many prompts are a…
-
### 🚀 The feature, motivation and pitch
I see that llama-stack is becoming very powerful set of tool which sits on top of LLM model.
inference, memory, agent, scoring, eval etc can be used via APIs…
-
xtuner train cfg-gamma7b/gemma_7b_it_qlora_alpaca_e3_copy.py --deepspeed deepspeed_zero2
10/28 18:19:05 - mmengine - WARNING - WARNING: command error: ''Adafactor is already registered in optimizer a…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
llama-index: 0.10.62
Python 3.11.9
Hi Llama-Index Community!
I think I am messing…
-
**Bug Description**
I'm using huggingface as the provider to generate feedback from a RAG model that uses TruLlama as the base of the feedback recorder. Even though I'm using _record.wait_for_feedbac…
-
For this code section using `ChatMistralAI` and `MistralAIEmbeddings`
```python
from langchain_ollama.chat_models import ChatOllama
from langchain_ollama.embeddings import OllamaEmbeddings
import …
-
`CUDA_VISIBLE_DEVICES=0,1 lm_eval --model vllm \
--model_args pretrained=/home/jovyan/data-vol-1/models/meta-llama__Llama3.1-70B-Instruct,tensor_parallel_size=2,dtype=auto,gpu_memory_utilization=…
-
Traceback (most recent call last):
File "/home/yunyi/container/whisper_ru/Whisper-Finetune-master/finetune.py", line 171, in
main()
File "/home/yunyi/container/whisper_ru/Whisper-Finetune-…
-
In the code line ‘python run_eval.py --eval_config_path config/tekgen_vicuna_config.json’, when I replace the configuration file, not all 7 configuration files in the config directory can run. Specifi…