search

omerbt / TokenFlow

Official Pytorch Implementation for "TokenFlow: Consistent Diffusion Features for Consistent Video Editing" presenting "TokenFlow" (ICLR 2024)
https://diffusion-tokenflow.github.io
MIT License
1.52k stars 134 forks source link

Visualising diffusion features. #19

Closed lmm077 closed 10 months ago

lmm077 commented 10 months ago

Would you release the code of visualising diffusion features (Fig 2 in the paper)?

hyoseok1223 commented 7 months ago

@lmm077 Hi, did you get diffusion feature visualization? I want do it, but I can't find extract tokens, compute NN field and propagation. it looks just use pnp instead. Please give me more explain?

FatLong666 commented 3 months ago

@lmm077 I also couldn't find the NN field computation code and propagation, It looks like the normal SD denoising process.

Sutongtong233 commented 1 month ago

@lmm077 Hi, did you get diffusion feature visualization? I want do it, but I can't find extract tokens, compute NN field and propagation. it looks just use pnp instead. Please give me more explain?

compute NN field is just a simple cosine similarity computation, here https://github.com/omerbt/TokenFlow/blob/8ae24e9d00069ffec24407a60f1d410f43035393/tokenflow_utils.py#L335 propagation is using torch.gather function to fetch corresponding feature from keyframe(one or two): https://github.com/omerbt/TokenFlow/blob/8ae24e9d00069ffec24407a60f1d410f43035393/tokenflow_utils.py#L372 extract tokens is not explicitly shown in the code. In function make_tokenflow_attention_block, TokenFlowBlock cover each block in Unet, and hidden_states(tokens) is directly as input.