Open loretoparisi opened 2 years ago
if you have the K80 and want to add it to the supported list, run :
!pip install git+https://github.com/facebookresearch/xformers@51dd119#egg=xformers
after around 40min, and the installation is done, navigate to /usr/local/lib/python3.7/dist-packages/xformers
save the two files : "_C_flashattention.so" and "_C.so", upload them to any host and send me the link and I will integrate them in the Colab for K80 users.
the files might not show in the colab explorer, so you will have to rename them
!cp /usr/local/lib/python3.7/dist-packages/xformers/_C.so /usr/local/lib/python3.7/dist-packages/xformers/C.py
!cp /usr/local/lib/python3.7/dist-packages/xformers/_C_flashattention.so /usr/local/lib/python3.7/dist-packages/xformers/C_flashattention.py
Hi. I am using collab pro and very often I have a100 gpu assigned. How can you add to the gpu ?
In a few hours it will be added to the colabs
Perhaps this is the right issue. Running on A100 (colab), getting spammed in the output:
FATAL: this function is for sm80, but was built for sm600
did you make a clean run with an update colab from the repo ?
did you make a clean run with an update colab from the repo ?
Not sure, will try it now and report back.
... or perhaps not now, but when I get A100 again 🤷♂️
@TheLastBen , actually, that doesn't seem to be A100-specific issue.
On V100 I see:
FATAL: this function is for sm70, but was built for sm600
Running freshly opened: https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb
Is it a regression?
Will try to reproduce on clean AUTOMATIC1111
UI without attention patch.
UPD: confirmed, clean AUTOMATIC1111
works on V100, the issue is introduced by the patch.
try to reproduce the error with a T4 (free colab)
also try other colabs if the same issue happens
save the two files : "_C_flashattention.so" and "_C.so", upload them to any host and send me the link and I will integrate them in the Colab for K80 users
I did this, but am using conda, so the directories don't match up. I found /home/ben/.conda/envs/my-env/lib/python3.10/site-packages/xformers
, but there's no _C_flashattention.so
in it, or anywhere on the system.
I did find _C.so
, though; it's here
For the K80 ? 287 KB looks a bit small, it should be at least 19mb, maybe it didn't compile well. try compiling it with google colab if you get the K80
For the K80 ? 287 KB looks a bit small, it should be at least 19mb, maybe it didn't compile well. try compiling it with google colab if you get the K80
I'll let you know if I can get the files from a K80 colab. Right now all my accounts have a usage limit of no GPU, or are getting T4s, though, so it might be a while.
@TheLastBen I was able to install xformers with a bigger _C.so
file (13.1 MB) by un-init
-ing conda, and got this. I still don't have a _C_flashattention.so
anywhere on my system, though.
GPUs unsupported by flash attention don't produce a _C_flashattention.so
after compiling, but they still benefit from a speed increase
GPUs unsupported by flash attention don't produce a
_C_flashattention.so
after compiling, but they still benefit from a speed increase
Ok, thanks. Quick question: what is _C_flashattention.so
for/what does it do?
The C/C++/Cuda code responsible for the xformers-specific operations (memory efficient attention included) for the underlying machine (python version, cuda, ..)
@TheLastBen Would you be able to make the whl using that file and add it? I'd make the whl and do a new PR, but it's not working for me.
Sorry, I completely forgot about it, I'll add it as soon as I'm done with the new Dreambooth method
@TheLastBen Would you be able to make the whl using that file and add it? I'd make the whl and do a new PR, but it's not working for me.
Just added, if you get the K80 try it in A1111 Colab and let me know if the wheel works.
Hello, from the notebook I can see that pre-compiled are available only for
T4
,P100
andV100
gpus:What. about other gpu like K80? Thanks.