Closed chenchunhui97 closed 4 months ago
try if you can import flash_attn separately:
>>> import flash_attn
>>> # will hit some error here?
some releated issue: https://github.com/Dao-AILab/flash-attention/issues/919
import flash_attn
Python 3.10.14 (main, Mar 21 2024, 16:24:04) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import flash_attn
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'flash_attn'
seems I missed to compile and install flash_attntion? but I compile follwing the instruction in portal. how to compile and install it additionaly?
try if you can import flash_attn separately:
>>> import flash_attn >>> # will hit some error here?
some releated issue: Dao-AILab/flash-attention#919
thanks i solved the promble by the releate issue. I installed flash_attn v2.5.6.
git clone https://github.com/Dao-AILab/flash-attention
cd flash_attn
git checkout v2.5.6
python3 setup.py bdist_wheel --dist-dir=dist
then compile and install flash_attn.
Your current environment
install:
pip install -e .
orpython3 setup.py bdist_wheel --dist-dir=dist
then run:to launch
How would you like to use vllm
I found I canot use Flashattention backend when install it from source file.
INFO 04-25 18:42:52 selector.py:74] Cannot use FlashAttention backend because the flash_attn package is not found. Please install it for better performance.