-
### System Info
transformers 4.43.3, python 3.10, linux
### Who can help?
@ArthurZucker
### Information
- [ ] The official example scripts
- [X] My own modified scripts
### Tasks
- [ ] An offi…
-
### System Info / 系統信息
乌班图v24,docker启动
2080ti*4
cuda12.6
Driver Version: 560.31.02
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
- [X] docker / docker
- [ ] pip install / 通过 p…
-
The below PostTrainingQuantConfig produces fp32 ops for NPU using 2.4.1. Models with int8 and fp16 ops would be preferred for NPU.
conf=PostTrainingQuantConfig(quant_level='auto',
device='n…
-
**Describe the bug**
I am using the `quant_with_alpaca.py` script to quantize MaziyarPanahi/Llama-3-70B-Instruct-32k-v0.1. I am using the following command:
``` python
python quant_with_alpaca.p…
-
# Todo
https://github.com/OCA/maintainer-tools/wiki/Migration-to-version-17.0
# Modules to migrate
- [x] delivery_procurement_group_carrier - By @peluko00 - #1570
- [x] purchase_stock_picki…
-
```
from transformers import Qwen2VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from qwen_vl_utils import process_vision_info
from transformers import Qwen2VLProcessor
from awq.models.q…
-
```
Some parameters are on the meta device device because they were offloaded to the cpu.
Quantizing weights: 0%| | 0/1771 [00:00
-
When I use aimet autoquant to quant my model, I met the following issues:
- Prepare Model
Traceback (most recent call last):
File "/workspace/aimet/build/staging/universal/lib/python/aimet_torch/…
-
系统ubuntu20.01。paddle和slim均是dev版本,CUDA11.6,cudnn8.4,按照官方说明这个是匹配的,但是执行自动压缩的时候,还是报版本不匹配,这是咋回事?
2023-06-01 15:00:54,113-INFO: devices: gpu
2023-06-01 15:01:03,250-INFO: Selected strategies: ['qat_dis']
…
-
Hi guys!
Love your work but I bought another GPU recently for the release of Llama 3 and hit a wall.
**Description**
Like a lot of small wallet devs, I have a dual RTX 3090.
I was expecting to u…
Skit5 updated
1 month ago