Installation problem

[liangbingzhao]

Thx for your great work! I wonder if it is possible to upgrade diffusers to 0.11.0. Many current works need updated diffusers. Really appreciate.

[daemon]

Yep, will do.

[liangbingzhao]

I downgrade my diffusers and try to use daam. But encountered this error:

Traceback (most recent call last):
  File "/root/picasso/zlb/anaconda3/envs/tuneplus/lib/python3.10/site-packages/daam/trace.py", line 116, in compute_global_heat_map
    maps = torch.stack(all_merges, dim=0)
RuntimeError: stack expects a non-empty TensorList

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/picasso/zlb/anaconda3/envs/tuneplus/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/picasso/zlb/anaconda3/envs/tuneplus/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/root/.vscode-server/extensions/ms-python.python-2022.16.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/root/picasso/zlb/Tune-A-Video/infer_daam.py", line 19, in <module>
    heat_map = tc.compute_global_heat_map()
  File "/root/picasso/zlb/anaconda3/envs/tuneplus/lib/python3.10/site-packages/daam/trace.py", line 121, in compute_global_heat_map
    raise RuntimeError('No heat maps found. Did you forget to call `with trace(...)` during generation?')
RuntimeError: No heat maps found. Did you forget to call `with trace(...)` during generation?

I just use the code in your README, test on stable diffusion v1-4, can you help to figure out what's the problem?

[daemon]

Hmm, interesting. What's your pip list?

[liangbingzhao]

here is my pip list:

Package                       Version
----------------------------- -----------------
absl-py                       1.4.0
accelerate                    0.16.0
aiofiles                      23.1.0
aiohttp                       3.8.4
aiosignal                     1.3.1
altair                        4.2.2
antlr4-python3-runtime        4.9.3
anyio                         3.6.2
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
asttokens                     2.2.1
async-timeout                 4.0.2
attrs                         22.2.0
backcall                      0.2.0
backports.functools-lru-cache 1.6.4
beautifulsoup4                4.11.2
bitsandbytes                  0.35.4
bleach                        6.0.0
blis                          0.7.9
cachetools                    5.3.0
catalogue                     2.0.8
certifi                       2022.12.7
cffi                          1.15.1
charset-normalizer            3.0.1
click                         8.1.3
cmake                         3.25.2
comm                          0.1.2
confection                    0.0.4
contourpy                     1.0.7
cycler                        0.11.0
cymem                         2.0.7
daam                          0.0.12
debugpy                       1.6.6
decorator                     5.1.1
decord                        0.6.0
defusedxml                    0.7.1
diffusers                     0.11.1
einops                        0.6.0
entrypoints                   0.4
executing                     1.2.0
fastapi                       0.92.0
fastjsonschema                2.16.2
ffmpy                         0.3.0
filelock                      3.9.0
flit_core                     3.8.0
fonttools                     4.38.0
frozenlist                    1.3.3
fsspec                        2023.1.0
ftfy                          6.1.1
google-auth                   2.16.0
google-auth-oauthlib          0.4.6
gradio                        3.18.0
grpcio                        1.51.1
h11                           0.14.0
httpcore                      0.16.3
httpx                         0.23.3
huggingface-hub               0.12.0
idna                          3.4
imageio                       2.25.0
importlib-metadata            6.0.0
importlib-resources           5.10.2
inflect                       6.0.2
ipykernel                     6.21.2
ipython                       8.10.0
ipython-genutils              0.2.0
ipywidgets                    8.0.4
jedi                          0.18.2
Jinja2                        3.1.2
joblib                        1.2.0
jsonschema                    4.17.3
jupyter                       1.0.0
jupyter_client                8.0.2
jupyter-console               6.5.1
jupyter_core                  5.2.0
jupyter-events                0.6.3
jupyter_server                2.3.0
jupyter_server_terminals      0.4.4
jupyterlab-pygments           0.2.2
jupyterlab-widgets            3.0.5
kiwisolver                    1.4.4
langcodes                     3.3.0
linkify-it-py                 1.0.3
lit                           15.0.7
llvmlite                      0.39.1
lmdb                          1.4.0
Markdown                      3.4.1
markdown-it-py                2.1.0
MarkupSafe                    2.1.2
matplotlib                    3.7.0
matplotlib-inline             0.1.6
mdit-py-plugins               0.3.3
mdurl                         0.1.2
mistune                       2.0.5
modelcards                    0.1.6
multidict                     6.0.4
murmurhash                    1.0.9
mypy-extensions               1.0.0
nbclassic                     0.5.1
nbclient                      0.7.2
nbconvert                     7.2.9
nbformat                      5.7.3
nest-asyncio                  1.5.6
networkx                      3.0
nltk                          3.8.1
notebook                      6.5.2
notebook_shim                 0.2.2
numba                         0.56.4
numpy                         1.23.5
nvidia-cublas-cu11            11.10.3.66
nvidia-cuda-nvrtc-cu11        11.7.99
nvidia-cuda-runtime-cu11      11.7.99
nvidia-cudnn-cu11             8.5.0.96
oauthlib                      3.2.2
omegaconf                     2.3.0
opencv-python                 4.7.0.68
orjson                        3.8.6
packaging                     23.0
pandas                        1.5.3
pandocfilters                 1.5.0
parso                         0.8.3
pathy                         0.10.1
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        9.4.0
pip                           23.0
pkgutil_resolve_name          1.3.10
platformdirs                  3.0.0
ply                           3.11
preshed                       3.0.8
prometheus-client             0.16.0
prompt-toolkit                3.0.36
protobuf                      4.21.12
psutil                        5.9.4
ptyprocess                    0.7.0
pure-eval                     0.2.2
pyasn1                        0.4.8
pyasn1-modules                0.2.8
pycparser                     2.21
pycryptodome                  3.17
pydantic                      1.10.4
pydub                         0.25.1
Pygments                      2.14.0
pyparsing                     3.0.9
PyQt5                         5.15.7
PyQt5-sip                     12.11.0
pyre-extensions               0.0.23
pyrsistent                    0.19.3
python-dateutil               2.8.2
python-json-logger            2.0.6
python-multipart              0.0.5
pytz                          2022.7.1
PyWavelets                    1.4.1
PyYAML                        6.0
pyzmq                         25.0.0
qtconsole                     5.4.0
QtPy                          2.3.0
regex                         2022.10.31
requests                      2.28.2
requests-oauthlib             1.3.1
rfc3339-validator             0.1.4
rfc3986                       1.5.0
rfc3986-validator             0.1.1
rsa                           4.9
scikit-image                  0.19.3
scipy                         1.10.0
Send2Trash                    1.8.0
setuptools                    67.1.0
sip                           6.7.7
six                           1.16.0
smart-open                    6.3.0
sniffio                       1.3.0
soupsieve                     2.3.2.post1
spacy                         3.5.0
spacy-legacy                  3.0.12
spacy-loggers                 1.0.4
srsly                         2.4.5
stack-data                    0.6.2
starlette                     0.25.0
tensorboard                   2.12.0
tensorboard-data-server       0.7.0
tensorboard-plugin-wit        1.8.1
terminado                     0.17.1
thinc                         8.1.7
tifffile                      2023.2.3
tinycss2                      1.2.1
tokenizers                    0.13.2
toml                          0.10.2
toolz                         0.12.0
torch                         1.13.1
torchaudio                    0.13.1
torchvision                   0.14.1
tornado                       6.2
tqdm                          4.64.1
traitlets                     5.9.0
transformers                  4.24.0
triton                        2.0.0.dev20230208
typer                         0.7.0
typing_extensions             4.4.0
typing-inspect                0.8.0
uc-micro-py                   1.0.1
urllib3                       1.26.14
uvicorn                       0.20.0
wasabi                        1.1.1
wcwidth                       0.2.6
webencodings                  0.5.1
websocket-client              1.5.1
websockets                    10.4
Werkzeug                      2.2.2
wheel                         0.38.4
widgetsnbextension            4.0.5
xformers                      0.0.17.dev444
yarl                          1.8.2
zipp                          3.13.0

When I test, I downgrade diffusers to 0.9.0

[daemon]

Ah, it's because xformers exists, so diffusers tries to use a different attention op. Try pip uninstall xformers for now.

https://github.com/huggingface/diffusers/blob/v0.9.0/src/diffusers/models/attention.py#L429

[liangbingzhao]

so for now, I couldn't use daam when using xformers? Would it be possible to enable xformers in the future?

[daemon]

Unfortunately not, you could also disable xformers from being used in that memory op. It should be possible in the future.

[daemon]

Updated just now.

• Doesn't break if xformers is installed.
• Works with latest Diffusers 0.14.0 and Transformers 4.27 libraries.