-
The [Quantization aware training in Keras example](https://www.tensorflow.org/model_optimization/guide/quantization/training_example?hl=en) mentions the following after performing quantization aware t…
-
1. Is it possible to add quantization? with ai toolkit i get 22.1gb vram doing 1024 training, and 1400 steps done in 45 mins
in kohya 1024 training, takes me nearly 2 hours with the same amount.
…
-
Hi,
I am trying to use Pytorch's native QAT instead of pytorch_nndct and then use Vitis AI's Quantization and Compilation for a VCK190. Is there a way to do this? If not would the new ONNX compati…
-
### 🚀 The feature, motivation and pitch
I'm trying to use float8 to try mistral nemo which was trained in a quantization aware way, to do inference in FP8.
Ref:
- https://mistral.ai/news/mistral-…
-
Thank you for the amazing work. I was able to setup the BEVFusion inference using the model files given in the readme.
I want to use this pipeline for BEVFusion trained on my dataset, so as per the […
-
When I run the following command to fine-tune Quantized BERT on MRPC,
nlp-train transformer_glue \
--task_name mrpc \
--model_name_or_path bert-base-uncased \
--model_type quant_bert \…
-
Hi, I am training a model using quantization-aware training, and I have a couple of questions:
1.
It seems to actually increase the size of the model (from ~87Mb to ~137Mb). I have come across th…
-
When I try to convert a custom model to tflite model, I am getting all nan
```
[{'name': 'input_1', 'index': 0, 'shape': array([ 1, 416, 416, 3], dtype=int32), 'shape_signature': array([ -1, 41…
-
### Report of performance regression
Hi I use this:
```
server_vllm.py \
--model "/data/models_temp/functionary-small-v2.4/" \
--served-model-name "functionary" \
--dtype=bfloat16 \
-…
rvsh2 updated
1 month ago
-
## ❓ Questions
Hi, when I try to reproduce the training code based on your released part, I meet a question when I try to use multiple-GPU to train, that is, I find that [https://github.com/facebo…