search

kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
Apache License 2.0
741 stars 39 forks source link

Missing pip packages flash_attn and wheel #69

Closed bitbottrap closed 2 months ago

bitbottrap commented 2 months ago

Might want to add those two packages to requirements file.

UnicornChan commented 2 months ago

Flash-attn is an optional package, and not all models require it. Additionally, if a precompiled package for flash-attn is not available, it needs to be compiled and installed, which can be very time-consuming. Therefore, flash-attn has not been added to the requirements.

We apologize for the omission of wheel package. We will include it in the next release. We use environments created with conda, which include wheel by default, so we did not encounter this issue.

bitbottrap commented 2 months ago

Sound reasoning. Might want to document the possibility of needing flash_attn. Great project!