-
Prior to filing: check that this should be a bug instead of a feature request. Everything supported, including the compatible versions of TensorFlow, is listed in the overview page of each technique. …
-
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current Behavior
D:\Anaconda\envs\langchain\python.exe E:/langchain-ChatGLM-master/cli_demo.py
D:\Anaconda\en…
-
### Your current environment
The output of `python collect_env.py`
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A…
-
`# Text Encoder
text_encoder = T5EncoderModel.from_pretrained(model_path, subfolder="text_encoder", torch_dtype=weight_dtype).to(device)
quantize_(text_encoder, quantization())
# Tran…
-
### 🚀 The feature, motivation and pitch
Please consider adding support for GPTQ and AWQ quantized Mixtral models.
I guess that after #4012 it's technically possible.
### Alternatives
_No r…
-
Hi, a few weeks ago @morettif and I finetuned the `Llama70B` with QLoRA on a H100:
- `r=32`
- `alpha=64`
- `quantize=bnb.nf4-dq`
- `precision=bf16-true`
- `weight_decay=0`
- `batch_size=32`
-…
-
Hello @edgchen1 @wejoncy I tried to quantize the mars-model used in deepsort tracking. Using the example in `image_classification/cpu ` I am able to quantize my mars model. Size of the model has reduc…
-
I have a 3070Ti and was wondering if running this training pipeline on consumer grade hardware is possible. If not, then what is the recommend hardware requirement and cost of training?
-
## ❓ InternalError when running llava model
Im new to mlc-llm and I'm not sure if this is a bug or me doing something incorrectly. I have so far not managed to run any model successfully. I have tr…
plufz updated
1 month ago
-
Hi, I am trying to run the `Llama-3.1 8b + Unsloth 2x faster finetuning.ipynb` you provided in the README. However, when I use google colab to run the second cell I got this error:
``` bash
------…