search

quic / aimet

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
https://quic.github.io/aimet-pages/index.html
Other
2.14k stars 382 forks source link

Export model by using Aimet for AI stack stable diffusion demo #2973

Open kurtqiao opened 5 months ago

kurtqiao commented 5 months ago

==Background tring to export model for AI stack stable diffusion demo, by following: https://docs.qualcomm.com/bundle/publicresource/topics/80-64748-1/model_updates.html

==Failure

failure on step: Apply AIMET Adaptive Rounding (AdaRound) to the TE model ModuleNotFoundError: No module named 'aimet_common.AimetTensorQuantizer' any idea about this? i'm using torch 1.9.1+cu111，AimetCommon tf-gpu-1.25.0，AimetTorch torch-gpu-1.25.0 any other version should i try ?

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[5], line 3
      1 import json
      2 from argparse import Namespace
----> 3 from aimet_quantsim import apply_adaround_te, calibrate_te
      5 with open('config.json', 'rt') as f:
      6     config = Namespace(**json.load(f))

File /home/aipc/Downloads/80-64748-2/aimet_quantsim.py:27
      1 # ==============================================================================
      2 #  @@-COPYRIGHT-START-@@
      3 #
   (...)
     23 #  @@-COPYRIGHT-END-@@
     24 # ==============================================================================
     26 import os
---> 27 import exceptions
     28 from tqdm.auto import tqdm
     30 import torch

File /home/aipc/Downloads/80-64748-2/exceptions.py:30
      1 # ==============================================================================
      2 #  @@-COPYRIGHT-START-@@
      3 #
   (...)
     25 
     26 # write custom exception/overrides functions here
     28 import torch
---> 30 from aimet_torch.qc_quantize_op import QcQuantizeWrapper
     31 from aimet_torch import elementwise_ops
     33 from redefined_modules.diffusers.models.attention import CrossAttention, AttentionBlock

File /usr/local/lib/python3.8/dist-packages/aimet_torch/qc_quantize_op.py:54
     52 from aimet_torch import utils
     53 from aimet_torch.tensor_factory_utils import constant_tensor_factory
---> 54 from aimet_torch.tensor_quantizer import StaticGridPerTensorQuantizer, StaticGridPerChannelQuantizer, TensorQuantizer, \
     55     LearnedGridTensorQuantizer, ParameterQuantizer
     56 import aimet_torch.quantsim_straight_through_grad as ste
     58 _logger = AimetLogger.get_area_logger(AimetLogger.LogAreas.Quant)

File /usr/local/lib/python3.8/dist-packages/aimet_torch/tensor_quantizer.py:46
     42 from typing import List, Union, Tuple, Optional
     44 import torch
---> 46 import aimet_common.AimetTensorQuantizer as AimetTensorQuantizer
     47 import aimet_common.libpymo as libpymo
     48 from aimet_common.defs import QuantScheme, QuantizationDataType, MAP_QUANT_SCHEME_TO_PYMO

ModuleNotFoundError: No module named 'aimet_common.AimetTensorQuantizer'

==My Test Environment Test DUT OS: Ubuntu 22.04 PRETTY_NAME="Ubuntu 22.04.4 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.4 LTS (Jammy Jellyfish)"

with GPU ---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.179 Driver Version: 535.179 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce RTX 4060 ... Off | 00000000:01:00.0 Off | N/A | | N/A 45C P0 N/A / 80W | 8MiB / 8188MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| +---------------------------------------------------------------------------------------+

Docker: running docker by following the tutorial: docker run --rm --gpus all --name=aimet-dev-torch-gpu -v $PWD:$PWD -w $PWD -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --network=host --ulimit core=-1 --ipc=host --shm-size=8G --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -it artifacts.codelinaro.org/codelinaro-aimet/aimet-dev:latest.torch-gpu

Docker environment cat /etc/os-release NAME="Ubuntu" VERSION="20.04.6 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.6 LTS"

my pip list

Package Version

absl-py 2.1.0 AimetCommon tf-gpu-1.25.0 AimetTorch torch-gpu-1.25.0 alabaster 0.7.13 anyio 4.3.0 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 astroid 2.5.3 asttokens 2.4.1 async-lru 2.0.4 atomicwrites 1.4.1 attrs 23.2.0 autograd 1.6.2 Babel 2.14.0 backcall 0.2.0 beautifulsoup4 4.12.3 behave 1.2.6 bleach 6.1.0 blosc 1.10.1 bokeh 1.2.0 cachetools 4.2.4 ccimport 0.4.2 certifi 2024.2.2 cffi 1.12.3 charset-normalizer 3.3.2 clarabel 0.7.1 click 8.1.7 cma 2.7.0 coloredlogs 15.0.1 comm 0.2.2 contourpy 1.1.1 coverage 7.4.4 cumm-cu111 0.2.8 cvxpy 1.4.2 cycler 0.12.1 cylp 0.92.2 Cython 0.29.12 dataclasses 0.6 debugpy 1.8.1 decorator 5.1.1 defusedxml 0.7.1 Deprecated 1.2.14 diffusers 0.10.2 docutils 0.16 ecos 2.0.13 exceptiongroup 1.2.0 executing 2.0.1 fastjsonschema 2.19.1 filelock 3.13.3 fire 0.6.0 flatbuffers 24.3.25 fonttools 4.50.0 fqdn 1.5.1 fsspec 2024.3.1 ftfy 6.2.0 future 1.0.0 git-pylint-commit-hook 2.6.1 google-auth 1.35.0 google-auth-oauthlib 0.4.6 graphviz 0.20.3 grpcio 1.62.1 grpcio-tools 1.48.2 h11 0.14.0 h5py 2.10.0 holoviews 1.12.7 httpcore 1.0.5 httpx 0.27.0 huggingface-hub 0.22.2 humanfriendly 10.0 hvplot 0.4.0 idna 3.6 imagesize 1.4.1 importlib_metadata 7.1.0 importlib_resources 6.4.0 ipykernel 6.29.4 ipython 8.12.3 ipywidgets 8.1.2 isoduration 20.11.0 isort 4.3.21 jax 0.4.13 jedi 0.19.1 Jinja2 3.0.3 joblib 1.3.2 json5 0.9.24 jsonpointer 2.4 jsonschema 4.21.1 jsonschema-specifications 2023.12.1 jupyter 1.0.0 jupyter_client 8.6.1 jupyter-console 6.6.3 jupyter_core 5.7.2 jupyter-events 0.10.0 jupyter-lsp 2.2.4 jupyter_server 2.13.0 jupyter_server_terminals 0.5.3 jupyterlab 4.1.5 jupyterlab_pygments 0.3.0 jupyterlab_server 2.25.4 jupyterlab_widgets 3.0.10 Keras 2.2.4 Keras-Applications 1.0.8 Keras-Preprocessing 1.1.2 kiwisolver 1.4.5 lark 1.1.9 lazy-object-proxy 1.10.0 lmdb 1.2.1 Markdown 3.6 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.7.5 matplotlib-inline 0.1.6 mccabe 0.6.1 mdurl 0.1.2 meld 3.20.2 mistune 3.0.2 ml-dtypes 0.2.0 mock 5.1.0 more-itertools 10.2.0 mpmath 1.3.0 nbclient 0.10.0 nbconvert 7.16.3 nbformat 5.10.3 nbsphinx 0.8.12 nest-asyncio 1.6.0 netron 7.5.7 networkx 3.1 ninja 1.11.1.1 notebook 7.1.2 notebook_shim 0.2.4 numpy 1.23.5 oauthlib 3.2.2 onnx 1.13.0 onnxruntime_extensions 0.10.1 onnxruntime-gpu 1.15.1 onnxsim 0.4.10 opencv-python 4.9.0.80 opt-einsum 3.3.0 osqp 0.6.5 overrides 7.7.0 packaging 24.0 pandas 1.4.3 pandocfilters 1.5.1 param 1.13.0 parse 1.20.1 parse-type 0.6.2 parso 0.8.3 pccm 0.4.11 pexpect 4.9.0 pickleshare 0.7.5 pillow 10.3.0 Pillow-SIMD 9.0.0.post1 pip 24.0 pkgutil_resolve_name 1.3.10 platformdirs 4.2.0 pluggy 0.12.0 portalocker 2.8.2 progressbar2 4.4.2 prometheus_client 0.20.0 prompt-toolkit 3.0.43 protobuf 3.20.1 psutil 5.9.8 ptflops 0.7.2.2 ptyprocess 0.7.0 pure-eval 0.2.2 py 1.11.0 pyasn1 0.6.0 pyasn1_modules 0.4.0 pybind11 2.12.0 pycairo 1.16.2 pycparser 2.22 pyDOE2 1.3.0 pydot 2.0.0 Pygments 2.17.2 PyGObject 3.36.0 pylint 2.3.1 pymoo 0.4.1 pyparsing 3.1.2 pytest 4.6.5 pytest-cov 2.6.1 python-dateutil 2.9.0.post0 python-json-logger 2.0.7 python-utils 3.8.2 pytorch-ignite 0.5.0.post2 pytz 2024.1 pyviz_comms 3.0.2 PyYAML 6.0.1 pyzmq 25.1.2 qdldl 0.1.7.post0 qtconsole 5.5.1 QtPy 2.4.1 referencing 0.34.0 regex 2023.12.25 requests 2.31.0 requests-oauthlib 2.0.0 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rich 13.7.1 rpds-py 0.18.0 rsa 4.9 sacremoses 0.1.1 scikit-learn 1.1.3 scipy 1.8.1 scs 3.2.4.post1 Send2Trash 1.8.2 setuptools 41.0.1 six 1.16.0 sniffio 1.3.1 snowballstemmer 2.2.0 soupsieve 2.5 spconv-cu111 2.1.20 Sphinx 2.1.1 sphinx-autodoc-typehints 1.6.0 sphinx-jinja 1.1.1 sphinx-rtd-theme 1.3.0 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.1 sphinxcontrib-jquery 4.1 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.5 stack-data 0.6.3 sympy 1.12 tensorboard 2.4.0 tensorboard-plugin-wit 1.8.1 tensorboardX 2.4 termcolor 2.4.0 terminado 0.18.1 threadpoolctl 3.4.0 timm 0.4.12 tinycss2 1.2.1 tokenizers 0.11.1 tomli 2.0.1 torch 1.9.1+cu111 torchaudio 0.10.1+cu111 torchinfo 1.8.0 torchtext 0.14.1 torchvision 0.10.1+cu111 tornado 6.4 tqdm 4.66.2 traitlets 5.14.2 transformers 4.23.1 types-python-dateutil 2.9.0.20240316 typing_extensions 4.10.0 uri-template 1.3.0 urllib3 2.2.1 wcwidth 0.2.13 webcolors 1.13 webencodings 0.5.1 websocket-client 1.7.0 Werkzeug 3.0.1 wget 3.2 wheel 0.33.4 widgetsnbextension 4.0.10 wrapt 1.12.1 zipp 3.18.1

kurtqiao commented 5 months ago

anyone has any comments, i think this should be a torch & AimetCommon version compatible issue