How to generate visual image of 'Similarity of Attention Outputs' as shown in Figure 4

thu-nics / DiTFastAttn

MIT License

94 stars 8 forks source link

How to generate visual image of 'Similarity of Attention Outputs' as shown in Figure 4 #6

Open nankepan opened 2 months ago

nankepan commented 2 months ago

Hi, Very nice work! How to generate visual image of 'Similarity of Attention Outputs' as shown in Figure 4? Could the author share the code for visualization??

nankepan commented 2 months ago

One more question: Is the cosine similarity calculation based on flattened latent or other ways? Could the author share the calculation code? I would like to verify the redundancies mentioned in the paper. Thanks!

Probe100 commented 2 months ago

We have added the code to calculate and visualize the similarity of attention outputs in experiments/cfg_similarity_experiments/. Please comment below if you encounter any issues while running the code or if you have further questions about the output.