ImportError: cannot import name 'ActivationCasting' from 'torchao.float8.inference' in cli_demo_quantization.py

I set up new project today and built diffusers as instructed. Normal inference working fine Working fine python inference\cli_demo.py --prompt "A girl riding a bike." --model_path THUDM/CogVideoX1.5-5b --generate_type "t2v"

As mentioned in ReadMe, torchao is installed using

git clone https://github.com/pytorch/ao
cd ao
pip install -e .

The quantized version is still not working

(C:\ai\CogVideo\cv_venv) C:\ai\CogVideo>python inference\cli_demo_quantization.py --prompt "A girl riding a bike." --model_path THUDM/CogVideoX1.5-5b --quantization_scheme fp8 --dtype bfloat16
Traceback (most recent call last):
  File "C:\ai\CogVideo\inference\cli_demo_quantization.py", line 26, in <module>
    from torchao.float8.inference import ActivationCasting, QuantConfig, quantize_to_float8
ImportError: cannot import name 'ActivationCasting' from 'torchao.float8.inference' (C:\ai\CogVideo\cv_venv\Lib\site-packages\torchao\float8\inference.py)

(C:\ai\CogVideo\cv_venv) C:\ai\CogVideo>pip list
Package                 Version
----------------------- ------------
absl-py                 2.0.0
accelerate              1.1.1
aiofiles                23.2.1
aiohttp                 3.9.1
aiosignal               1.3.1
annotated-types         0.6.0
anyascii                0.3.2
anyio                   4.6.2.post1
attrs                   23.1.0
audioread               3.0.1
Babel                   2.14.0
bangla                  0.0.2
blinker                 1.7.0
blis                    0.7.11
bnnumerizer             0.0.2
bnunicodenormalizer     0.1.6
boto3                   1.35.63
botocore                1.35.63
braceexpand             0.1.7
cachetools              5.3.2
catalogue               2.0.10
certifi                 2023.11.17
cffi                    1.16.0
charset-normalizer      3.4.0
click                   8.1.7
cloudpathlib            0.16.0
colorama                0.4.6
confection              0.1.4
contourpy               1.2.0
coqpit                  0.0.17
cpm-kernels             1.0.11
cycler                  0.12.1
cymem                   2.0.8
Cython                  3.0.7
datasets                3.1.0
dateparser              1.1.8
decorator               4.4.2
deepspeed               0.15.0
diffusers               0.32.0.dev0
dill                    0.3.8
distro                  1.9.0
docopt                  0.6.2
einops                  0.8.0
encodec                 0.1.1
fastapi                 0.115.5
ffmpy                   0.4.0
filelock                3.13.1
Flask                   3.0.0
fonttools               4.47.0
frozenlist              1.4.1
fsspec                  2023.12.2
g2pkk                   0.1.2
google-auth             2.25.2
google-auth-oauthlib    1.2.0
gradio                  5.6.0
gradio_client           1.4.3
gruut                   2.2.3
gruut-ipa               0.13.0
gruut-lang-de           2.0.0
gruut-lang-en           2.0.0
gruut-lang-es           2.0.0
gruut-lang-fr           2.0.2
h11                     0.14.0
hangul-romanize         0.1.0
hjson                   3.1.0
httpcore                1.0.7
httpx                   0.27.2
huggingface-hub         0.26.2
idna                    3.6
imageio                 2.36.0
imageio-ffmpeg          0.5.1
importlib_metadata      8.5.0
inflect                 7.0.0
itsdangerous            2.1.2
jamo                    0.4.1
jieba                   0.42.1
Jinja2                  3.1.2
jiter                   0.7.1
jmespath                1.0.1
joblib                  1.3.2
jsonlines               1.2.0
kiwisolver              1.4.5
langcodes               3.3.0
llvmlite                0.41.1
Markdown                3.5.1
markdown-it-py          3.0.0
MarkupSafe              2.1.3
matplotlib              3.8.2
mdurl                   0.1.2
moviepy                 1.0.3
mpmath                  1.3.0
msgpack                 1.0.7
multidict               6.0.4
multiprocess            0.70.16
murmurhash              1.0.10
networkx                2.8.8
ninja                   1.11.1.1
nltk                    3.8.1
num2words               0.5.13
numba                   0.58.1
numpy                   1.26.0
nvidia-ml-py            12.560.30
oauthlib                3.2.2
openai                  1.54.4
orjson                  3.10.11
packaging               24.2
pandas                  1.5.3
pillow                  11.0.0
pip                     24.2
platformdirs            4.1.0
pooch                   1.8.0
preshed                 3.0.9
proglog                 0.1.10
protobuf                5.28.3
psutil                  5.9.7
py-cpuinfo              9.0.0
pyarrow                 18.0.0
pyasn1                  0.5.1
pyasn1-modules          0.3.0
pycparser               2.21
pydantic                2.9.2
pydantic_core           2.23.4
pydub                   0.25.1
Pygments                2.18.0
pynndescent             0.5.11
pyparsing               3.1.1
pypinyin                0.50.0
pysbd                   0.3.4
python-crfsuite         0.9.10
python-dateutil         2.8.2
python-multipart        0.0.12
pytz                    2023.3.post1
PyYAML                  6.0.1
regex                   2023.10.3
requests                2.32.3
requests-oauthlib       1.3.1
rich                    13.9.4
rsa                     4.9
ruff                    0.7.4
s3transfer              0.10.3
safehttpx               0.1.1
safetensors             0.4.5
scikit-learn            1.3.2
scikit-video            1.1.11
scipy                   1.14.1
semantic-version        2.10.0
sentencepiece           0.2.0
setuptools              75.1.0
shellingham             1.5.4
six                     1.16.0
smart-open              6.4.0
sniffio                 1.3.1
soundfile               0.12.1
soxr                    0.3.7
spacy                   3.7.2
spacy-legacy            3.0.12
spacy-loggers           1.0.5
srsly                   2.4.8
starlette               0.41.2
SudachiDict-core        20230927
SudachiPy               0.6.8
SwissArmyTransformer    0.4.12
sympy                   1.13.1
tensorboard-data-server 0.7.2
tensorboardX            2.6.2.2
thinc                   8.2.2
threadpoolctl           3.2.0
tokenizers              0.20.3
tomlkit                 0.12.0
torch                   2.5.1+cu121
torchao                 0.7.0+gitd4ca98f6 C:\ai\CogVideo\ao
torchvision             0.20.1+cu121
tqdm                    4.67.0
trainer                 0.0.36
transformers            4.46.2
TTS                     0.22.0
typer                   0.13.0
typing_extensions       4.12.2
tzdata                  2023.3
tzlocal                 5.2
umap-learn              0.5.5
Unidecode               1.3.7
urllib3                 2.2.3
uvicorn                 0.32.0
wasabi                  1.1.2
weasel                  0.3.4
webdataset              0.2.100
websockets              12.0
Werkzeug                3.0.1
wheel                   0.44.0
xxhash                  3.5.0
yarl                    1.9.4
zipp                    3.21.0

THUDM / CogVideo

ImportError: cannot import name 'ActivationCasting' from 'torchao.float8.inference' in cli_demo_quantization.py #509

System Info / 系統信息

Information / 问题信息

Reproduction / 复现过程

Expected behavior / 期待表现