search

facebookresearch / xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.
https://facebookresearch.github.io/xformers/
Other
8.66k stars 614 forks source link

Specific patch version of Python is required? #859

Open mrorii opened 1 year ago

mrorii commented 1 year ago

❓ Questions and Help

Hello, thanks for developing this great library!

I'm currently trying to install xformers on an AWS Deep Learning Container, but when running python -m xformers.info to check the installation, it looks like it fails due to incompatible Python versions and was wondering whether the pip wheels target a specific Python patch version.

sh-4.2$ docker run --gpus all --rm -it --entrypoint bash ${AWS_ACCOUNT}.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker
root@b14714a85bd2:/# pip install xformers==0.0.21
Collecting xformers==0.0.21
  Obtaining dependency information for xformers==0.0.21 from https://files.pythonhosted.org/packages/c7/b4/9f8bea4204f8482c9c9c64bcf86bd209ccfb2ebdb27e3590ef4c1e87b743/xformers-0.0.21-cp310-cp310-manylinux2014_x86_64.whl.metadata
  Downloading xformers-0.0.21-cp310-cp310-manylinux2014_x86_64.whl.metadata (1.0 kB)
Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (from xformers==0.0.21) (1.22.4)
Requirement already satisfied: torch==2.0.1 in /opt/conda/lib/python3.10/site-packages (from xformers==0.0.21) (2.0.1+cu118)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from torch==2.0.1->xformers==0.0.21) (3.12.3)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.10/site-packages (from torch==2.0.1->xformers==0.0.21) (4.7.1)
Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch==2.0.1->xformers==0.0.21) (1.12)
Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch==2.0.1->xformers==0.0.21) (3.1)
Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch==2.0.1->xformers==0.0.21) (3.1.2)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch==2.0.1->xformers==0.0.21) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch==2.0.1->xformers==0.0.21) (1.3.0)
Downloading xformers-0.0.21-cp310-cp310-manylinux2014_x86_64.whl (167.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 167.0/167.0 MB 12.6 MB/s eta 0:00:00
Installing collected packages: xformers
Successfully installed xformers-0.0.21
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@b14714a85bd2:/# python -m xformers.info
WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
    PyTorch 2.0.1+cu118 with CUDA 1108 (you have 2.0.1+cu118)
    Python  3.10.12 (you have 3.10.9)
  Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
  Memory-efficient attention, SwiGLU, sparse and more won't be available.
  Set XFORMERS_MORE_DETAILS=1 for more details
xFormers 0.0.21
memory_efficient_attention.cutlassF:               unavailable
memory_efficient_attention.cutlassB:               unavailable
memory_efficient_attention.decoderF:               unavailable
memory_efficient_attention.flshattF@0.0.0:         unavailable
memory_efficient_attention.flshattB@0.0.0:         unavailable
memory_efficient_attention.smallkF:                unavailable
memory_efficient_attention.smallkB:                unavailable
memory_efficient_attention.tritonflashattF:        unavailable
memory_efficient_attention.tritonflashattB:        unavailable
indexing.scaled_index_addF:                        unavailable
indexing.scaled_index_addB:                        unavailable
indexing.index_select:                             unavailable
swiglu.dual_gemm_silu:                             unavailable
swiglu.gemm_fused_operand_sum:                     unavailable
swiglu.fused.p.cpp:                                not built
is_triton_available:                               True
is_functorch_available:                            False
pytorch.version:                                   2.0.1+cu118
pytorch.cuda:                                      available
gpu.compute_capability:                            8.6
gpu.name:                                          NVIDIA A10G
build.info:                                        available
build.cuda_version:                                1108
build.python_version:                              3.10.12
build.torch_version:                               2.0.1+cu118
build.env.TORCH_CUDA_ARCH_LIST:                    5.0+PTX 6.0 6.1 7.0 7.5 8.0+PTX 9.0
build.env.XFORMERS_BUILD_TYPE:                     Release
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS:        None
build.env.NVCC_FLAGS:                              None
build.env.XFORMERS_PACKAGE_FROM:                   wheel-v0.0.21
source.privacy:                                    open source
root@b14714a85bd2:/# python -V
Python 3.10.9

I'm not sure where the build.python_version of 3.10.12 is coming from, even though I'm using python 3.10.9. If I want to use xformers 0.0.21 with python 3.10.9, do I need to install from source?

mrorii commented 1 year ago

If I want to use xformers 0.0.21 with python 3.10.9, do I need to install from source?

Following up on this, I verified that after installing from source, the output of python -m xformers.info no longer contains the xFormers can't load C++/CUDA extensions warning:

root@0b7da6dd0c4a:/# python -m xformers.info
xFormers 0.0.21+320b5ad.d20230915
memory_efficient_attention.cutlassF:               available
memory_efficient_attention.cutlassB:               available
memory_efficient_attention.decoderF:               available
memory_efficient_attention.flshattF@v2.0.8:        available
memory_efficient_attention.flshattB@v2.0.8:        available
memory_efficient_attention.smallkF:                available
memory_efficient_attention.smallkB:                available
memory_efficient_attention.tritonflashattF:        unavailable
memory_efficient_attention.tritonflashattB:        unavailable
indexing.scaled_index_addF:                        available
indexing.scaled_index_addB:                        available
indexing.index_select:                             available
swiglu.dual_gemm_silu:                             available
swiglu.gemm_fused_operand_sum:                     available
swiglu.fused.p.cpp:                                available
is_triton_available:                               True
is_functorch_available:                            False
pytorch.version:                                   2.0.1+cu118
pytorch.cuda:                                      available
gpu.compute_capability:                            8.6
gpu.name:                                          NVIDIA A10G
build.info:                                        available
build.cuda_version:                                1108
build.python_version:                              3.10.9
build.torch_version:                               2.0.1+cu118
build.env.TORCH_CUDA_ARCH_LIST:                    3.7 5.0 7.0+PTX 7.5+PTX 8.0
build.env.XFORMERS_BUILD_TYPE:                     None
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS:        None
build.env.NVCC_FLAGS:                              None
build.env.XFORMERS_PACKAGE_FROM:                   None
build.nvcc_version:                                11.8.89
source.privacy:                                    open source