-
Hi, when I use the command for evaluating Llama-2 7B on wikitext2:
lm_eval --model hf --model_args pretrained=meta-llama/Llama-2-7b-hf --tasks wikitext --device cuda:0 --batch_size 1
…
-
[meta engineering blog post](https://engineering.fb.com/2024/06/12/data-infrastructure/training-large-language-models-at-scale-meta/)
- Meta requires massive computational power to train large lang…
-
- Laravel-mongodb Version: 4.2.0
- PHP Version: 8.3.4
- Database Driver & Version: php8.3-mongodb latest
### Description:
I have a model with a mysql collection e.g.
$product. with a meta-field…
-
I have fine tuned "meta-llama-3.1-8b-bnb-4bit" model using unsloth. I have downloaded the lora weights and able to do inferencing using those on Colab GPU.
But i want use this fine tuned model for …
-
## Problem
Currently, the UI makes it impossible to tell if a model has finished streaming its response back to the user, or if it is still underway and is just taking a long time to calculate the re…
-
### Your current environment
vllm version: 0.5.4
gpu 24GB memory
### 🐛 Describe the bug
```bash
CUDA_VISIBLE_DEVICES=0 vllm serve mistralai/Mistral-7B-Instruct-v0.3 --api-key yyy --port 1…
-
### What is the issue?
I've had this issue for a while with earlier version of ollama and latest with and Intel SPR 8480+ and RTX 4090. The num_gpu parameter has been removed from model file so I can…
-
### System Info
I got this error when I tried to use sentiment classification pipeline with "nvidia/Mistral-NeMo-Minitron-8B-Base". It works fine with llama 3.1.
TypeError: MistralForSequenceClas…
-
### Your current environment
vllm v0.5.4
Setup A) single docker container with vllm, no pipeline-parallelism
```
docker run ... vllm/vllm-openai:v0.5.4 --model "meta-llama/Meta-Llama-3.1-70B-…
-
### What is the issue?
I tried 1xH100 box and got an error during installation. Got the same output from another bigger 2xH100 box too:
```
root@C.11391672:~$ curl -fsSL https://ollama.com/instal…