-
When running `examples/quantization/basic_usage_gpt_xl.py` an error occurs during the model packing:
```
2023-05-22 04:08:34 INFO [auto_gptq.quantization.gptq] duration: 0.16880011558532715
2023-…
-
Hi, I'm trying to run Llamaspeak following the Instructions on https://www.jetson-ai-lab.com/tutorial_llamaspeak.html
Specs:
Jetson Orin NX(16GB) Developer Kit
Jetpack 6.0 [L4T 36.3.0]
The RI…
-
I have the following error when i run this model. Can you tell me which pytorch version to use to run this model.
I have the following error
/home/nvidia/.local/lib/python3.8/site-packages/torch…
-
Prior to filing: check that this should be a bug instead of a feature request. Everything supported, including the compatible versions of TensorFlow, is listed in the overview page of each technique. …
-
Only q4_0_4_4 gguf are running in my Poco X6 pro phone. CPU-Z said it have cortex A510 and A715 cores. They are support both i8mm and sve. When i tried to run a gguf what needs it this happens:
~/…
-
The model downloaded from https://github.com/fatihcakirs/mobile_models/blob/main/v0_7/tflite/mobilebert_int8_384_20200602.tflite
Some Fully-connected weights has none-zero zero point (ex. weight `b…
-
Before you open an issue, please make sure you have tried the following steps:
1. Make sure your **environment** is the same with (https://mace.readthedocs.io/en/latest/installation/env_requirement…
wcfzl updated
5 years ago
-
Describe Question:
when I use W16A16 quantization in main branch, I cann't get qdq_error and xmodel_err, because I found that qdq_error and xmodel_err are all empty. I eventually found that model.g…
-
I've created a "real-time" application with chunks of 3 sec using my own small fine-tuned model. It reads the complete audio call recording and generate 3s chunks, but in some cases I get repetition o…
-
Currently if the user wants to have the fused model while preserving the original, they have to do
```python
import copy
import torchvision.models.quantization as models
model = models.resnet1…
z-a-f updated
5 years ago