-
Hey guys,
Does vLLM support the 4-bit quantized version of the Mixtral-8x7B-Instruct-v0.1 model downloaded from Hugging Face here https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1.
Acc…
-
### Describe the bug
Hi,
I have been trying to build a ColumnTransformer with different values in the n_jobs' parameter, but when fitting and transforming throws the error ValueError: cannot set…
-
Hey wonderful creator. In November last year I took a deep dive into wine. I have been trying to find out if a fully equipped media creator & 3D Linux Workstation would be in reach for myself, but mor…
-
I am trying to quantize one model with two input,
The model definition:
```python
class my_model(nn.Module):
def forward(left_input, right_input):
........
```
The dataset o…
-
Hello guys,
Congrats for the wonderful package / paper.
I ma just curious, before implementing this in OpenNMT-y, if you have somewhere some speed benchmark in tok/sec with other methods given a…
-
do i need to build the project using build_omim.sh -s to build android app in android studio
-
I am trying to fine tune gemma7-b with 4 A100 80 GB gpus using 4-bit qunatization
model_id = "google/gemma-7b"
# BitsAndBytesConfig int-4 config
bnb_config = BitsAndBytesConfig(
load_in_4b…
-
### System Info
**Hardware details**
CPU - AMD Ryzen Threadripper PRO 3955WX 16-Cores
GPU - NVIDIA RTX 4090
**Software details**
OS - Ubuntu 22.04.3 LTS
CUDA - 12.1 (I've also tried with 11.…
-
Hi, Arctic team, Great work! I followed the [Huggingface Inference Tutorial](https://github.com/Snowflake-Labs/snowflake-arctic/tree/main/inference) to do the inference. But I met the following error:…
JF-D updated
5 months ago
-
I downloaded ```autoawq-0.2.2+cu118-cp310-cp310-win_amd64.whl``` in releases but failed to install it.
The error message is as follows。
```
ERROR: autoawq_kernels-0.0.5+cu118-cp310-cp310-windows_am…