-
## 🚀 Feature
I'd like to be able to train iterable-style datasets instead of just map-style datasets.
(a map-style dataset in PyTorch has `__getitem__` and `__len__`, whereas iterable-style datasets…
-
The experimental results in the paper are set for MACs. In my opinion, quantization should not affect MACs. MACs refers to the number of operations of a multiplication and addition. How can smoothquan…
-
**Is your feature request related to a problem? Please describe.**
With PR #358 we might introduce ImageSharp, and derivates (ImageSharp.Drawing).
It makes sense to describe the way to go forward wi…
-
**Describe the bug**
ilab model train fails on Mac with InstructLab version 0.19 with "ValueError: too many values to unpack".
**To Reproduce**
1. ilab --version
ilab, version 0.19.0
2. % i…
-
Running the following command on a fresh torchchat install (81fce9c377822525b67f46f84b6f64b187fa8659, 6/26/2024) produces etSocketAddressDIRECTORYDIRECTORY instead of tokens.
The following command:…
-
### 🐛 Describe the bug
## Description
The output of fully quantized and fake quantized models do not match, with the fully quantized model not matching the expected analytical results for a minima…
-
### System Info
### TL;DR:
1. Quantization for the `lm_head` was fake-quantized, at least with `int4-awq` and `int8_sq` configurations. Model was Gemma-2b, Gemma-7b and Llama-2-7b. How can I mak…
-
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
### Describe the bug
When running the w4a16…
-
### System Info
CPU architecture : 86_64
CPU/Host memory size 187GB
GPU properties
GPU name : A10
GPU memory size : 24 GB
Clock frequencies used (if applicable)
Libraries
TensorRT-LLM tag …
-
I'm trying to quantize the JAIS model, but I received the message TypeError: **JAIS isn't supported yet**. This is my code:
```
from transformers import AutoTokenizer, TextGenerationPipeline
fr…