search

NVIDIA / TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.
https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/index.html
Apache License 2.0
1.6k stars 255 forks source link

TransformerEngine setup.py fails with Python 3.8 #958

Closed skydoorkai closed 6 days ago

skydoorkai commented 1 week ago

This is due to /build_tools/utils.py: from functools import cache

which is not a Python 3.8 feature.

CUDACXX=/usr/local/cuda/bin/nvcc NVTE_FRAMEWORK=pytorch python setup.py bdist_wheel

Traceback (most recent call last): File "setup.py", line 13, in from build_tools.build_ext import CMakeExtension, get_build_ext File "/w/github/TransformerEngine/build_tools/build_ext.py", line 20, in from .utils import ( File "/w/github/TransformerEngine/build_tools/utils.py", line 13, in from functools import cache ImportError: cannot import name 'cache' from 'functools' (/opt/conda/lib/python3.8/functools.py)

timmoon10 commented 1 week ago

Thanks for the catch, can you try with https://github.com/NVIDIA/TransformerEngine/pull/967?

skydoorkai commented 6 days ago

Thanks for the catch, can you try with #967?

This works for 3.8.