Add verbose weights

[rockerBOO]

python lora-inspector.py -v /mnt/900/lora/boo_v2.safetensors

down_block_0_proj_in weight average magnitude: 1.4924503799105646
down_block_0_proj_in weight average strength: 0.017584813472947592
down_block_0_proj_out weight average magnitude: 1.5157774537695803
down_block_0_proj_out weight average strength: 0.017854483676551985
unet_down_block_0_transformer_block_0_attentions_to_k weight average magnitude: 1.462021831711949
unet_down_block_0_transformer_block_0_attentions_to_k weight average strength: 0.014734332086778964
unet_down_block_0_transformer_block_0_attentions_to_out_0 weight average magnitude: 1.5096795979790278
unet_down_block_0_transformer_block_0_attentions_to_out_0 weight average strength: 0.017774835308342455
unet_down_block_0_transformer_block_0_attentions_to_q weight average magnitude: 1.445551043693953
unet_down_block_0_transformer_block_0_attentions_to_q weight average strength: 0.017098382122298614
unet_down_block_0_transformer_block_0_attentions_to_v weight average magnitude: 1.4463436289083411
unet_down_block_0_transformer_block_0_attentions_to_v weight average strength: 0.01456201074246142

[rockerBOO]

Trying to figure out the best way to break up the blocks. Also need to look into how to identify the block appropriately. There are tools out there with in/mid/out "layers" or something like in DAAM and block weight merging. But the keys do not have these relevant associations so I'm not sure of the connections there.

This representation may be appropriate and then some cleaning up of what values should be together. Attention weights (k, v, q) are tricky to try and group separately as they associate in a process. Maybe each block should just be evaluated together in most cases and maybe a very very verbose version breaking down each individual component (like to_k, to_v, to_q, ,and so on) averages.

Also not sure if its even a good idea to break this up into blocks. But maybe this will be good enough for some specific purpose.

Let me know what you think.

[rockerBOO]

• Verbose weights needs to include Text Encoder and Conv blocks as well.
• Clarify unet naming on all appropriate keys