-
1*8H100 DGX BOX
Torch version: 2.1.1
CUDA version: 12.1
VLLM: 0.2.3
The inference works just fine in tensor parallel 1 but when using **tp > 1** I am getting this error below:
WARNING 12-0…
-
`CUDA_VISIBLE_DEVICES=0,1 lm_eval --model vllm \
--model_args pretrained=/home/jovyan/data-vol-1/models/meta-llama__Llama3.1-70B-Instruct,tensor_parallel_size=2,dtype=auto,gpu_memory_utilization=…
-
### System Info
PyTorch version: 2.4.0+cu121
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### 🐛 Describe the bug
I was fine-tuning SLAM on my own da…
-
**Is your feature request related to a problem? Please describe.**
We should provide an interface for structural pruning methods, such as N
pruning based on weight magnitude or methods like Wanda,…
-
The CodeMMLU is a great piece of work!
I noticed that the dataset provides task_id, question, and choices columns, but is there an answer column?
How should I handle this dataset if I want to f…
-
We updated: enabled OCR and changed Top k to 40. We used the "Generative AI and the Nature of Work" paper and it still hallucinated 3 quotes.
This ticket is to have a conversation between D3 and AM.
…
-
Hi team, thanks for open source this awesome tool. I am new to the tool and try to ask some questions on LLM evaluation
1. Seems `evaluate` already create some evaluators (Some libs call it tasks I…
-
# URL
- https://arxiv.org/abs/2408.03314
# Authors
- Charlie Snell
- Jaehoon Lee
- Kelvin Xu
- Aviral Kumar
# Abstract
- Enabling LLMs to improve their outputs by using more test-time comput…
-
**Summary**
This paper investigates the relationship between temperature and cherry blossom bloom duration in Japan using a dual model approach. By combining historical and modern data from satellite…
-
**Describe the bug**
Running tests for Knowledge Retention (following the documentation: [https://docs.confident-ai.com/docs/metrics-knowledge-retention]) generates error: TypeError: Claude.generate(…