search

castorini / daam

Diffusion attentive attribution maps for interpreting Stable Diffusion.
MIT License
689 stars 63 forks source link

Fix merge indice calculation #16

Closed sALTaccount closed 1 year ago

sALTaccount commented 1 year ago

If you were to try to use the prompt hololive, amane_kanata, 1girl, hairclip, angel_wings, blue eyes on the old code, it wouldn't work. It assumes that all words that end up becoming two tokens won't be separated by , however that isn't the case. For example, 1girl becomes ['1</w>', 'girl</w>'], amanekanata becomes `['am', 'ane', '', 'kan', 'ata'], while something like hairclip becomes['hair', 'clip']`

I rewrote the token merge indice calculation to fix this inconsistencies, and it no longer depends on whether the token ends with </w> to determine if it is part of the word.

Here are some images of the new code working where the previous would fail:

I also made sure to check this to make sure it doesn't mess up when the search term is hair and hairclip is in the prompt, or if the same word is repeated in the prompt.

sALTaccount commented 1 year ago

I didn't realize this, but while I was making this fix, it looks like the main issue was fixed by the new commit. It used to not generate if something like 1girl was in the prompt, because it would offset the words and token list. My code still allows a heat map to be computed for an entire word rather than just a single token, although I don't know if that is the behavior you intended or not.

sALTaccount commented 1 year ago

Yeah, nevermind. This code isn't compatible with the new version. Unfortunate timing lol