-
llava multimodel would be huge to be supported for aws neuron chips
https://huggingface.co/llava-hf/llava-v1.6-mistral-7b-hf
This in particular is trending
I'm not sure if this is the correct…
-
### OS
Linux
### GPU Library
CUDA 12.x
### Python version
3.10
### Pytorch version
2.4.0
### Model
_No response_
### Describe the bug
I quantize my own qwen 7B model and the return token is…
-
### Feature request
Currently, ORTModel has a hard dependency on `torch` and `transformers`. Could we make it such that this dependency is soft, and that Optimum + ORTModel can be used without PyTo…
-
I was trying to run fine-tune-dp.py under "research/synthetic-text-generation-with-DP" directory
Error occurs below:
Traceback (most recent call last):
File "/root/autodl-tmp/dp-transformers/re…
-
### Issues Policy acknowledgement
- [x] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)
### Willi…
-
Attempting to use this library on a **gfx1030** (6800XT) with the huggingface transformers results in:
```
python -m bitsandbytes
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++…
-
Hi,
I'm working on adapting this code to building a language model on a medical dataset. Before running the full model on the entire dataset, I'm running it on a sample. Specifically, I just get a si…
-
Cannot create models for VQ-GAN，There is no information about the VQ-GAN model in taming/modules.Importing VQModel from taming.models.vqgan with error。
-
This is my demo code:
```
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Initialize the tokenizer and model from the pretrained version on Hugging Face
tokenizer …
-
### Your current environment
```text
Python version: 3.12.3
PyTorch version: 2.3.1+cu121
```
### How you are installing vllm
```
pip install vllm
```
Building wheels for collected packages:…