search

kohya-ss / sd-scripts

Apache License 2.0
5.23k stars 866 forks source link

LORA Extract not doing anything #875

Open alenknight opened 1 year ago

alenknight commented 1 year ago

running the LORA extractor isn't doing anything. it used to work months ago... but haven't tried it till now.... command terminal isn't showing any errors, the progress bar runs and says successful... but when i apply the LORA in a model... nothing.

loading original SD model : C:/AI/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned.safetensors UNet2DConditionModel: 64, 8, 768, False, False loading u-net: loading vae: loading text encoder: loading tuned SD model : C:/AI/stable-diffusion-webui/models/Stable-diffusion/MLKyunnymgx/MLKyunymgx_filewords_person.ckpt UNet2DConditionModel: 64, 8, 768, False, False loading u-net: loading vae: loading text encoder: create LoRA network. base dim (rank): 256, alpha: 256 neuron dropout: p=None, rank dropout: p=None, module dropout: p=None apply LoRA to Conv2d with kernel size (3,3). dim (rank): 256, alpha: 256.0 create LoRA for Text Encoder: create LoRA for Text Encoder: 72 modules. create LoRA for U-Net: 278 modules. create LoRA network. base dim (rank): 256, alpha: 256 neuron dropout: p=None, rank dropout: p=None, module dropout: p=None apply LoRA to Conv2d with kernel size (3,3). dim (rank): 256, alpha: 256.0 create LoRA for Text Encoder: create LoRA for Text Encoder: 72 modules. create LoRA for U-Net: 278 modules. Text encoder is same. Extract U-Net only. calculating by svd 100%|████████████████████████████████████████████████████████████████████████████████| 278/278 [00:26<00:00, 10.45it/s] create LoRA network from weights create LoRA for Text Encoder: create LoRA for Text Encoder: 0 modules. create LoRA for U-Net: 278 modules. enable LoRA for text encoder enable LoRA for U-Net Loading extracted LoRA weights: LoRA weights are saved to: C:\AI\stable-diffusion-webui\models\Lora\Custom\MLKyunnymgx.safetensors

kohya-ss commented 1 year ago

You can verify the weights of the extracted LoRA with networks/check_lora_weights.py (see --help for usage). Could you please share the result of the script?

DreamFilmVFX commented 1 year ago

Hi, i've the exactly same result. i'll post here my verified weights in a couple of minutes, and i hope you can resolve it for both of us.

alenknight commented 1 year ago

You can verify the weights of the extracted LoRA with networks/check_lora_weights.py (see --help for usage). Could you please share the result of the script?

Oops. Sorry. Didn't see this till now (apologies was traveling abroad. I'll check and post a bit later today)

DreamFilmVFX commented 1 year ago

here are the resulting weights:

loading: D:/AiTools/stable-diffusion-webui/models/Lora/FabioKOOL_LORAs.safetensors number of LoRA modules: 556 lora_unet_down_blocks_0_attentions_0_proj_in.lora_down.weight,(128-320-1-1),0.04425150901079178,4.3550139139370003e-07 lora_unet_down_blocks_0_attentions_0_proj_in.lora_up.weight,(320-128-1-1),0.004554857034236193,9.920830734699848e-08 lora_unet_down_blocks_0_attentions_0_proj_out.lora_down.weight,(128-320-1-1),0.04454754292964935,6.031627322045097e-07 lora_unet_down_blocks_0_attentions_0_proj_out.lora_up.weight,(320-128-1-1),0.004330792464315891,1.2118771053337696e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-320),0.044410280883312225,5.4743480859542615e-08 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_k.lora_up.weight,(320-128),0.009329566732048988,8.818092851470283e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-320),0.043477922677993774,1.8650040374268428e-06 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(320-128),0.005445981863886118,1.9690953934059507e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-320),0.04436790198087692,1.3345068055059528e-06 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_q.lora_up.weight,(320-128),0.01031370460987091,2.0952337820290268e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-320),0.043586041778326035,2.2756266844226047e-06 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn1_to_v.lora_up.weight,(320-128),0.005934516899287701,1.2918269476358546e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028700144961476326,4.2049214243888855e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn2_to_k.lora_up.weight,(320-128),0.020510125905275345,2.0163312797194521e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-320),0.044500865042209625,1.3105440643812472e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(320-128),0.003236226039007306,2.0630304220503604e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-320),0.04445091262459755,2.0576360384438885e-06 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn2_to_q.lora_up.weight,(320-128),0.013324874453246593,2.637177658471046e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.027889309450984,1.955777406692505e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_attn2_to_v.lora_up.weight,(320-128),0.009715639986097813,2.8185743872199964e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-320),0.04428514838218689,5.250196863926249e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(2560-128),0.007167242467403412,3.027441408676168e-08 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_ff_net_2.lora_down.weight,(128-1280),0.021995384246110916,2.7008354663848877e-07 lora_unet_down_blocks_0_attentions_0_transformer_blocks_0_ff_net_2.lora_up.weight,(320-128),0.014639702625572681,4.278396318113664e-06 lora_unet_down_blocks_0_attentions_1_proj_in.lora_down.weight,(128-320-1-1),0.04446053132414818,1.3137706673660432e-06 lora_unet_down_blocks_0_attentions_1_proj_in.lora_up.weight,(320-128-1-1),0.004439886659383774,9.909533105201263e-08 lora_unet_down_blocks_0_attentions_1_proj_out.lora_down.weight,(128-320-1-1),0.04440793767571449,4.230664671922568e-07 lora_unet_down_blocks_0_attentions_1_proj_out.lora_up.weight,(320-128-1-1),0.006307999603450298,8.129503044074227e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-320),0.04451587051153183,1.6352271359210135e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn1_to_k.lora_up.weight,(320-128),0.008472656831145287,9.271587941839243e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-320),0.04401993006467819,2.566408511484042e-06 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(320-128),0.005442948546260595,2.3839608331854834e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-320),0.04454667121171951,1.7163989696200588e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn1_to_q.lora_up.weight,(320-128),0.009476786479353905,1.270955181098543e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-320),0.044046323746442795,1.2245811831235187e-06 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn1_to_v.lora_up.weight,(320-128),0.00633149454370141,3.9528760709117705e-08 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.0287509486079216,9.147915989160538e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn2_to_k.lora_up.weight,(320-128),0.017526526004076004,1.7089669199776836e-06 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-320),0.04447755962610245,3.3509596164549293e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(320-128),0.003083595307543874,9.223122532375783e-08 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-320),0.04455186426639557,2.288603297984082e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn2_to_q.lora_up.weight,(320-128),0.011661219410598278,2.810579644574318e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.027880249544978142,5.19677996635437e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_attn2_to_v.lora_up.weight,(320-128),0.009320437908172607,1.2131341975418763e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-320),0.044334083795547485,1.5638924821814726e-07 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(2560-128),0.0068584224209189415,8.089725866966546e-08 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_ff_net_2.lora_down.weight,(128-1280),0.02209978736937046,8.009374141693115e-08 lora_unet_down_blocks_0_attentions_1_transformer_blocks_0_ff_net_2.lora_up.weight,(320-128),0.016580525785684586,9.435393053536245e-07 lora_unet_down_blocks_0_downsamplers_0_conv.lora_down.weight,(128-320-3-3),0.014673188328742981,1.424923539161682e-07 lora_unet_down_blocks_0_downsamplers_0_conv.lora_up.weight,(320-128-1-1),0.012001405470073223,3.312765386453975e-07 lora_unet_down_blocks_0_resnets_0_conv1.lora_down.weight,(128-320-3-3),0.014753320254385471,6.705522537231445e-08 lora_unet_down_blocks_0_resnets_0_conv1.lora_up.weight,(320-128-1-1),0.013412552885711193,1.4064477227293537e-07 lora_unet_down_blocks_0_resnets_0_conv2.lora_down.weight,(128-320-3-3),0.014013287611305714,1.280568540096283e-07 lora_unet_down_blocks_0_resnets_0_conv2.lora_up.weight,(320-128-1-1),0.013708663173019886,3.3121739306807285e-07 lora_unet_down_blocks_0_resnets_0_time_emb_proj.lora_down.weight,(128-1280),0.02154352143406868,4.2491592466831207e-07 lora_unet_down_blocks_0_resnets_0_time_emb_proj.lora_up.weight,(320-128),0.005564338993281126,1.9424132347012346e-07 lora_unet_down_blocks_0_resnets_1_conv1.lora_down.weight,(128-320-3-3),0.01436504814773798,2.491287887096405e-08 lora_unet_down_blocks_0_resnets_1_conv1.lora_up.weight,(320-128-1-1),0.018129823729395866,1.762421447892848e-06 lora_unet_down_blocks_0_resnets_1_conv2.lora_down.weight,(128-320-3-3),0.014019065536558628,4.190951585769653e-08 lora_unet_down_blocks_0_resnets_1_conv2.lora_up.weight,(320-128-1-1),0.01515440084040165,1.7396082796494738e-07 lora_unet_down_blocks_0_resnets_1_time_emb_proj.lora_down.weight,(128-1280),0.021812070161104202,1.6624107956886292e-07 lora_unet_down_blocks_0_resnets_1_time_emb_proj.lora_up.weight,(320-128),0.005754522047936916,5.593656737801211e-07 lora_unet_down_blocks_1_attentions_0_proj_in.lora_down.weight,(128-640-1-1),0.03131614997982979,1.6995558738130967e-08 lora_unet_down_blocks_1_attentions_0_proj_in.lora_up.weight,(640-128-1-1),0.00595491286367178,2.6024077115494038e-08 lora_unet_down_blocks_1_attentions_0_proj_out.lora_down.weight,(128-640-1-1),0.03153573349118233,6.255847324609931e-07 lora_unet_down_blocks_1_attentions_0_proj_out.lora_up.weight,(640-128-1-1),0.004986128769814968,3.805380899279953e-08 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-640),0.03146306797862053,8.659114882902941e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn1_to_k.lora_up.weight,(640-128),0.011282146908342838,1.0807739592166854e-08 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-640),0.030781015753746033,3.225692069008801e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(640-128),0.006850868463516235,2.3826160600037838e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-640),0.03145509585738182,1.0687230087569333e-06 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn1_to_q.lora_up.weight,(640-128),0.011723374016582966,5.557071176554018e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-640),0.03105955757200718,9.331401429335529e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn1_to_v.lora_up.weight,(640-128),0.007374988403171301,3.697997641438633e-08 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.02863326668739319,6.807968020439148e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn2_to_k.lora_up.weight,(640-128),0.018091123551130295,8.468612833212319e-08 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-640),0.03075140155851841,6.18855040102062e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(640-128),0.005659658927470446,1.1613710171332059e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-640),0.03151736408472061,5.980469381938747e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn2_to_q.lora_up.weight,(640-128),0.01600894331932068,1.7944452679330425e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.02747090347111225,1.862645149230957e-08 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_attn2_to_v.lora_up.weight,(640-128),0.009022814221680164,1.4117281921244285e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-640),0.03141806647181511,8.564414599732117e-08 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(5120-128),0.00800537783652544,3.6893943455851286e-10 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_ff_net_2.lora_down.weight,(128-2560),0.015278467908501625,1.4971010386943817e-07 lora_unet_down_blocks_1_attentions_0_transformer_blocks_0_ff_net_2.lora_up.weight,(640-128),0.017330221831798553,1.7388772732829239e-07 lora_unet_down_blocks_1_attentions_1_proj_in.lora_down.weight,(128-640-1-1),0.03140236809849739,1.6627686818537768e-06 lora_unet_down_blocks_1_attentions_1_proj_in.lora_up.weight,(640-128-1-1),0.005130289122462273,1.1523720644390778e-08 lora_unet_down_blocks_1_attentions_1_proj_out.lora_down.weight,(128-640-1-1),0.03152795881032944,2.292443241458386e-06 lora_unet_down_blocks_1_attentions_1_proj_out.lora_up.weight,(640-128-1-1),0.005781872197985649,5.5513524443995266e-08 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-640),0.03152397274971008,3.3748600003491447e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn1_to_k.lora_up.weight,(640-128),0.010657668113708496,7.631235376948098e-08 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-640),0.030815517529845238,3.444467822077968e-08 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(640-128),0.006701372563838959,4.196556702140697e-08 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-640),0.03155461698770523,1.614880943634489e-06 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn1_to_q.lora_up.weight,(640-128),0.010975552722811699,2.2576091396331321e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-640),0.03125707060098648,7.923815132926393e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn1_to_v.lora_up.weight,(640-128),0.0070664407685399055,3.7220712556518265e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.02869650535285473,2.39349901676178e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn2_to_k.lora_up.weight,(640-128),0.016419947147369385,4.0035791926129605e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-640),0.03101367875933647,1.2475893651497927e-09 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(640-128),0.004710487090051174,6.071603309010243e-08 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-640),0.03154756873846054,1.8668259826881695e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn2_to_q.lora_up.weight,(640-128),0.015293890610337257,1.1578109848642271e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.027082063257694244,9.331852197647095e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_attn2_to_v.lora_up.weight,(640-128),0.008853369392454624,6.911108840768065e-08 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-640),0.0314878448843956,1.7719015943384875e-07 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(5120-128),0.00784575566649437,5.457630436467298e-08 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_ff_net_2.lora_down.weight,(128-2560),0.015503302216529846,6.126356311142445e-08 lora_unet_down_blocks_1_attentions_1_transformer_blocks_0_ff_net_2.lora_up.weight,(640-128),0.018013153225183487,7.917109314803383e-07 lora_unet_down_blocks_1_downsamplers_0_conv.lora_down.weight,(128-640-3-3),0.010426489636301994,3.1315721571445465e-08 lora_unet_down_blocks_1_downsamplers_0_conv.lora_up.weight,(640-128-1-1),0.014125737361609936,2.6434872779645957e-07 lora_unet_down_blocks_1_resnets_0_conv1.lora_down.weight,(128-320-3-3),0.014619807712733746,1.1391239240765572e-07 lora_unet_down_blocks_1_resnets_0_conv1.lora_up.weight,(640-128-1-1),0.016353607177734375,2.639810929849773e-07 lora_unet_down_blocks_1_resnets_0_conv2.lora_down.weight,(128-640-3-3),0.01027872134000063,2.153683453798294e-08 lora_unet_down_blocks_1_resnets_0_conv2.lora_up.weight,(640-128-1-1),0.022002199664711952,1.152995281472613e-07 lora_unet_down_blocks_1_resnets_0_conv_shortcut.lora_down.weight,(128-320-1-1),0.043760739266872406,4.1490829971735366e-06 lora_unet_down_blocks_1_resnets_0_conv_shortcut.lora_up.weight,(640-128-1-1),0.002717087510973215,5.803877378696143e-10 lora_unet_down_blocks_1_resnets_0_time_emb_proj.lora_down.weight,(128-1280),0.021651936694979668,2.5122426450252533e-07 lora_unet_down_blocks_1_resnets_0_time_emb_proj.lora_up.weight,(640-128),0.005420113913714886,4.402919628887503e-08 lora_unet_down_blocks_1_resnets_1_conv1.lora_down.weight,(128-640-3-3),0.010308315046131611,1.7462298274040222e-10 lora_unet_down_blocks_1_resnets_1_conv1.lora_up.weight,(640-128-1-1),0.021561861038208008,1.114948645408731e-06 lora_unet_down_blocks_1_resnets_1_conv2.lora_down.weight,(128-640-3-3),0.010019978508353233,4.6566128730773926e-09 lora_unet_down_blocks_1_resnets_1_conv2.lora_up.weight,(640-128-1-1),0.01885157823562622,7.280078762050834e-07 lora_unet_down_blocks_1_resnets_1_time_emb_proj.lora_down.weight,(128-1280),0.021727876737713814,4.353933036327362e-08 lora_unet_down_blocks_1_resnets_1_time_emb_proj.lora_up.weight,(640-128),0.004759740550071001,1.0146789719556182e-07 lora_unet_down_blocks_2_attentions_0_proj_in.lora_down.weight,(128-1280-1-1),0.02210046723484993,4.334137315709086e-08 lora_unet_down_blocks_2_attentions_0_proj_in.lora_up.weight,(1280-128-1-1),0.008846724405884743,1.5947891540690762e-07 lora_unet_down_blocks_2_attentions_0_proj_out.lora_down.weight,(128-1280-1-1),0.02230069600045681,4.458486841940612e-07 lora_unet_down_blocks_2_attentions_0_proj_out.lora_up.weight,(1280-128-1-1),0.007268681190907955,1.1462374516213458e-07 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-1280),0.022239331156015396,6.724490475562561e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn1_to_k.lora_up.weight,(1280-128),0.014738878235220909,8.830819098193388e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-1280),0.02184462919831276,3.928797411845153e-07 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(1280-128),0.009464427828788757,8.970055631607465e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-1280),0.022246437147259712,1.8603999762945023e-07 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn1_to_q.lora_up.weight,(1280-128),0.014840483665466309,3.828097021596477e-07 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-1280),0.0221390537917614,9.667189004858301e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn1_to_v.lora_up.weight,(1280-128),0.009321510791778564,2.271870300774026e-07 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.02871147356927395,4.947902993990283e-07 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn2_to_k.lora_up.weight,(1280-128),0.014501702971756458,6.257344864479819e-09 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-1280),0.02221151441335678,3.1554115764720336e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(1280-128),0.009988515637814999,2.4730683279017285e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-1280),0.022291650995612144,1.9501747772210365e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn2_to_q.lora_up.weight,(1280-128),0.016382262110710144,1.73973209172118e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.027184801176190376,2.497646391930175e-07 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_attn2_to_v.lora_up.weight,(1280-128),0.008932170458137989,3.749461185975633e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-1280),0.022278334945440292,1.4660811231692605e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(10240-128),0.01067371852695942,1.1237818675624567e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_ff_net_2.lora_down.weight,(128-5120),0.011041361838579178,1.7954288722421552e-08 lora_unet_down_blocks_2_attentions_0_transformer_blocks_0_ff_net_2.lora_up.weight,(1280-128),0.02239028550684452,7.440560523264139e-08 lora_unet_down_blocks_2_attentions_1_proj_in.lora_down.weight,(128-1280-1-1),0.02212887816131115,9.051913707480708e-07 lora_unet_down_blocks_2_attentions_1_proj_in.lora_up.weight,(1280-128-1-1),0.00911368615925312,4.7288406079815104e-08 lora_unet_down_blocks_2_attentions_1_proj_out.lora_down.weight,(128-1280-1-1),0.02229994721710682,8.154685104955206e-08 lora_unet_down_blocks_2_attentions_1_proj_out.lora_up.weight,(1280-128-1-1),0.007896050810813904,3.9141674790243997e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-1280),0.022273700684309006,5.014509696366076e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn1_to_k.lora_up.weight,(1280-128),0.015318053774535656,8.940627793663225e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-1280),0.02184186317026615,7.604027985053108e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(1280-128),0.010322878137230873,2.07462687029647e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-1280),0.022275257855653763,1.0741383249524006e-07 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn1_to_q.lora_up.weight,(1280-128),0.015327394008636475,1.0342009204578062e-07 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-1280),0.022201593965291977,5.4134947191641913e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn1_to_v.lora_up.weight,(1280-128),0.010343879461288452,1.7325916701338429e-07 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028741011396050453,3.253349234455527e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn2_to_k.lora_up.weight,(1280-128),0.015587246045470238,3.484343480408825e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-1280),0.02220110595226288,3.13149513431199e-07 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(1280-128),0.011761797592043877,3.157308370305145e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-1280),0.02228020876646042,4.5343838905864686e-07 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn2_to_q.lora_up.weight,(1280-128),0.018124159425497055,2.66061306319898e-07 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.02721053920686245,1.304948114011495e-07 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_attn2_to_v.lora_up.weight,(1280-128),0.009977961890399456,1.1677477829152849e-07 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-1280),0.022248754277825356,1.3988058356062538e-07 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(10240-128),0.011678756214678288,6.997880053205563e-09 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_ff_net_2.lora_down.weight,(128-5120),0.011039813980460167,1.4999564257323073e-08 lora_unet_down_blocks_2_attentions_1_transformer_blocks_0_ff_net_2.lora_up.weight,(1280-128),0.024681590497493744,5.39234179086634e-07 lora_unet_down_blocks_2_downsamplers_0_conv.lora_down.weight,(128-1280-3-3),0.007408184465020895,8.657530692346427e-09 lora_unet_down_blocks_2_downsamplers_0_conv.lora_up.weight,(1280-128-1-1),0.021728362888097763,7.056443678266078e-07 lora_unet_down_blocks_2_resnets_0_conv1.lora_down.weight,(128-640-3-3),0.010400519706308842,3.6465630515181147e-09 lora_unet_down_blocks_2_resnets_0_conv1.lora_up.weight,(1280-128-1-1),0.020584503188729286,3.155330716708704e-07 lora_unet_down_blocks_2_resnets_0_conv2.lora_down.weight,(128-1280-3-3),0.007360519375652075,7.101773835671565e-09 lora_unet_down_blocks_2_resnets_0_conv2.lora_up.weight,(1280-128-1-1),0.027383197098970413,2.3505491242303833e-07 lora_unet_down_blocks_2_resnets_0_conv_shortcut.lora_down.weight,(128-640-1-1),0.03080402873456478,3.3139454558295256e-07 lora_unet_down_blocks_2_resnets_0_conv_shortcut.lora_up.weight,(1280-128-1-1),0.0037313911598175764,4.628628857972217e-08 lora_unet_down_blocks_2_resnets_0_time_emb_proj.lora_down.weight,(128-1280),0.021228712052106857,9.34555046683272e-08 lora_unet_down_blocks_2_resnets_0_time_emb_proj.lora_up.weight,(1280-128),0.0033189826644957066,3.88373067039538e-08 lora_unet_down_blocks_2_resnets_1_conv1.lora_down.weight,(128-1280-3-3),0.007396080531179905,1.2494753676151049e-08 lora_unet_down_blocks_2_resnets_1_conv1.lora_up.weight,(1280-128-1-1),0.028835585340857506,3.890729516342617e-08 lora_unet_down_blocks_2_resnets_1_conv2.lora_down.weight,(128-1280-3-3),0.00734470272436738,6.517476780487641e-09 lora_unet_down_blocks_2_resnets_1_conv2.lora_up.weight,(1280-128-1-1),0.02741738222539425,1.0266271743830657e-07 lora_unet_down_blocks_2_resnets_1_time_emb_proj.lora_down.weight,(128-1280),0.021373486146330833,1.3686825184322515e-07 lora_unet_down_blocks_2_resnets_1_time_emb_proj.lora_up.weight,(1280-128),0.0030277136247605085,6.74739042416661e-10 lora_unet_down_blocks_3_resnets_0_conv1.lora_down.weight,(128-1280-3-3),0.007416050881147385,8.770576487293624e-10 lora_unet_down_blocks_3_resnets_0_conv1.lora_up.weight,(1280-128-1-1),0.025431448593735695,5.0008754470809436e-08 lora_unet_down_blocks_3_resnets_0_conv2.lora_down.weight,(128-1280-3-3),0.00738662201911211,3.2075875289194755e-09 lora_unet_down_blocks_3_resnets_0_conv2.lora_up.weight,(1280-128-1-1),0.025523077696561813,1.15441253001336e-06 lora_unet_down_blocks_3_resnets_0_time_emb_proj.lora_down.weight,(128-1280),0.021070925518870354,4.598415515033594e-08 lora_unet_down_blocks_3_resnets_0_time_emb_proj.lora_up.weight,(1280-128),0.0019806018099188805,3.098753031949286e-09 lora_unet_down_blocks_3_resnets_1_conv1.lora_down.weight,(128-1280-3-3),0.007420070935040712,1.9343172397157105e-08 lora_unet_down_blocks_3_resnets_1_conv1.lora_up.weight,(1280-128-1-1),0.02558264695107937,7.689317271797336e-08 lora_unet_down_blocks_3_resnets_1_conv2.lora_down.weight,(128-1280-3-3),0.007346377708017826,8.376063398429778e-09 lora_unet_down_blocks_3_resnets_1_conv2.lora_up.weight,(1280-128-1-1),0.02584812045097351,4.345054946952587e-07 lora_unet_down_blocks_3_resnets_1_time_emb_proj.lora_down.weight,(128-1280),0.021075818687677383,1.2695805651219416e-07 lora_unet_down_blocks_3_resnets_1_time_emb_proj.lora_up.weight,(1280-128),0.0018492592498660088,5.098452504626039e-08 lora_unet_mid_block_attentions_0_proj_in.lora_down.weight,(128-1280-1-1),0.022193530574440956,3.7767918570352776e-08 lora_unet_mid_block_attentions_0_proj_in.lora_up.weight,(1280-128-1-1),0.011596131138503551,5.7202186098948e-07 lora_unet_mid_block_attentions_0_proj_out.lora_down.weight,(128-1280-1-1),0.022296123206615448,3.2511977110516455e-07 lora_unet_mid_block_attentions_0_proj_out.lora_up.weight,(1280-128-1-1),0.010289412923157215,2.6049653545356932e-09 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-1280),0.022304745391011238,3.0814928209110803e-07 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn1_to_k.lora_up.weight,(1280-128),0.01340457983314991,1.2777722702139727e-07 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-1280),0.022234123200178146,9.042850734886088e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(1280-128),0.012180578894913197,3.6407277548278216e-07 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-1280),0.02229638583958149,6.939400236660731e-07 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn1_to_q.lora_up.weight,(1280-128),0.013443413190543652,5.8780162959237714e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-1280),0.022272780537605286,6.915266936857734e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn1_to_v.lora_up.weight,(1280-128),0.01164616271853447,8.205976342878785e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028782552108168602,3.1671152100898325e-07 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn2_to_k.lora_up.weight,(1280-128),0.013101942837238312,5.8676647540778504e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-1280),0.02218761295080185,3.519117797168292e-07 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(1280-128),0.011806374415755272,3.082034893964192e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-1280),0.022295858711004257,4.253548624433279e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn2_to_q.lora_up.weight,(1280-128),0.014351382851600647,3.537160608857448e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.02830660156905651,8.333360312917648e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_attn2_to_v.lora_up.weight,(1280-128),0.009988322854042053,6.034887434225311e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-1280),0.022286733612418175,1.530471962496449e-07 lora_unet_mid_block_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(10240-128),0.011893515475094318,2.3734433085564888e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_ff_net_2.lora_down.weight,(128-5120),0.011070295237004757,2.7834511584501342e-08 lora_unet_mid_block_attentions_0_transformer_blocks_0_ff_net_2.lora_up.weight,(1280-128),0.025396227836608887,3.56987972516265e-09 lora_unet_mid_block_resnets_0_conv1.lora_down.weight,(128-1280-3-3),0.007421921472996473,2.218527361819156e-09 lora_unet_mid_block_resnets_0_conv1.lora_up.weight,(1280-128-1-1),0.023183079436421394,1.3212808624984973e-08 lora_unet_mid_block_resnets_0_conv2.lora_down.weight,(128-1280-3-3),0.00729257520288229,4.8933728180600156e-09 lora_unet_mid_block_resnets_0_conv2.lora_up.weight,(1280-128-1-1),0.024394460022449493,9.391592925567238e-08 lora_unet_mid_block_resnets_0_time_emb_proj.lora_down.weight,(128-1280),0.021077577024698257,1.8834479931228998e-07 lora_unet_mid_block_resnets_0_time_emb_proj.lora_up.weight,(1280-128),0.0016302622389048338,6.2318008531292435e-09 lora_unet_mid_block_resnets_1_conv1.lora_down.weight,(128-1280-3-3),0.007429810240864754,1.0253908833135483e-08 lora_unet_mid_block_resnets_1_conv1.lora_up.weight,(1280-128-1-1),0.02317938208580017,1.5332038572068996e-07 lora_unet_mid_block_resnets_1_conv2.lora_down.weight,(128-1280-3-3),0.007288816384971142,6.3566951702398455e-09 lora_unet_mid_block_resnets_1_conv2.lora_up.weight,(1280-128-1-1),0.02396831288933754,1.982947566148141e-07 lora_unet_mid_block_resnets_1_time_emb_proj.lora_down.weight,(128-1280),0.021050382405519485,2.4390826069975446e-07 lora_unet_mid_block_resnets_1_time_emb_proj.lora_up.weight,(1280-128),0.0015820093685761094,1.865453125304839e-09 lora_unet_up_blocks_0_resnets_0_conv1.lora_down.weight,(128-2560-3-3),0.005226935725659132,2.84066281608375e-09 lora_unet_up_blocks_0_resnets_0_conv1.lora_up.weight,(1280-128-1-1),0.02765309438109398,2.872313018542627e-07 lora_unet_up_blocks_0_resnets_0_conv2.lora_down.weight,(128-1280-3-3),0.007189665921032429,1.1007492695114252e-09 lora_unet_up_blocks_0_resnets_0_conv2.lora_up.weight,(1280-128-1-1),0.024148032069206238,6.5630225698498634e-09 lora_unet_up_blocks_0_resnets_0_conv_shortcut.lora_down.weight,(128-2560-1-1),0.015554005280137062,2.0893125451948436e-08 lora_unet_up_blocks_0_resnets_0_conv_shortcut.lora_up.weight,(1280-128-1-1),0.00873769260942936,1.3916189089968611e-08 lora_unet_up_blocks_0_resnets_0_time_emb_proj.lora_down.weight,(128-1280),0.020995138213038445,2.0709529735540855e-07 lora_unet_up_blocks_0_resnets_0_time_emb_proj.lora_up.weight,(1280-128),0.0014712968841195107,8.71344063568813e-09 lora_unet_up_blocks_0_resnets_1_conv1.lora_down.weight,(128-2560-3-3),0.00522611103951931,2.637645102332442e-10 lora_unet_up_blocks_0_resnets_1_conv1.lora_up.weight,(1280-128-1-1),0.03306093066930771,9.914916176967381e-08 lora_unet_up_blocks_0_resnets_1_conv2.lora_down.weight,(128-1280-3-3),0.00733380438759923,1.884349343228564e-09 lora_unet_up_blocks_0_resnets_1_conv2.lora_up.weight,(1280-128-1-1),0.029327083379030228,3.2572825148236006e-07 lora_unet_up_blocks_0_resnets_1_conv_shortcut.lora_down.weight,(128-2560-1-1),0.015634853392839432,7.942990976061992e-08 lora_unet_up_blocks_0_resnets_1_conv_shortcut.lora_up.weight,(1280-128-1-1),0.007504045031964779,3.050645069890834e-09 lora_unet_up_blocks_0_resnets_1_time_emb_proj.lora_down.weight,(128-1280),0.020900476723909378,1.7327032253433572e-07 lora_unet_up_blocks_0_resnets_1_time_emb_proj.lora_up.weight,(1280-128),0.0017674925038591027,1.9304907894479584e-08 lora_unet_up_blocks_0_resnets_2_conv1.lora_down.weight,(128-2560-3-3),0.00522260507568717,3.133712622727103e-09 lora_unet_up_blocks_0_resnets_2_conv1.lora_up.weight,(1280-128-1-1),0.03543791547417641,4.606841486065605e-08 lora_unet_up_blocks_0_resnets_2_conv2.lora_down.weight,(128-1280-3-3),0.007339656352996826,5.025723393004e-09 lora_unet_up_blocks_0_resnets_2_conv2.lora_up.weight,(1280-128-1-1),0.028932467103004456,1.3462351944326656e-07 lora_unet_up_blocks_0_resnets_2_conv_shortcut.lora_down.weight,(128-2560-1-1),0.015653250738978386,1.3428447243057917e-09 lora_unet_up_blocks_0_resnets_2_conv_shortcut.lora_up.weight,(1280-128-1-1),0.006108303554356098,9.338197770603074e-08 lora_unet_up_blocks_0_resnets_2_time_emb_proj.lora_down.weight,(128-1280),0.02086462639272213,1.4981385731971386e-07 lora_unet_up_blocks_0_resnets_2_time_emb_proj.lora_up.weight,(1280-128),0.001809221925213933,3.093511224960821e-08 lora_unet_up_blocks_0_upsamplers_0_conv.lora_down.weight,(128-1280-3-3),0.007384799420833588,1.170909502690165e-08 lora_unet_up_blocks_0_upsamplers_0_conv.lora_up.weight,(1280-128-1-1),0.01702510379254818,1.8104259424944757e-07 lora_unet_up_blocks_1_attentions_0_proj_in.lora_down.weight,(128-1280-1-1),0.02218642458319664,6.395594596142473e-07 lora_unet_up_blocks_1_attentions_0_proj_in.lora_up.weight,(1280-128-1-1),0.01012221910059452,1.3746012328397228e-08 lora_unet_up_blocks_1_attentions_0_proj_out.lora_down.weight,(128-1280-1-1),0.022312121465802193,8.059136291649338e-08 lora_unet_up_blocks_1_attentions_0_proj_out.lora_up.weight,(1280-128-1-1),0.009691722691059113,1.9251605465342436e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-1280),0.02228866145014763,2.876082021430193e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn1_to_k.lora_up.weight,(1280-128),0.016426291316747665,2.4138128651429724e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-1280),0.02177595905959606,9.801699007994102e-08 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(1280-128),0.011816746555268764,1.989613140906954e-09 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-1280),0.02229083515703678,8.692231290297059e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn1_to_q.lora_up.weight,(1280-128),0.016420472413301468,7.153542469495733e-08 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-1280),0.022214284166693687,8.314630406403012e-08 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn1_to_v.lora_up.weight,(1280-128),0.012236559763550758,2.3255351777606847e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028685620054602623,1.9607325612014392e-06 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn2_to_k.lora_up.weight,(1280-128),0.015880927443504333,2.670370520263532e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-1280),0.02228226512670517,3.028183357400849e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(1280-128),0.013144440948963165,4.243744058385346e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-1280),0.022305911406874657,7.899026286395383e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn2_to_q.lora_up.weight,(1280-128),0.01866733655333519,3.542091633335076e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.027628475800156593,1.491814884957421e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_attn2_to_v.lora_up.weight,(1280-128),0.011442650109529495,1.6564638372074114e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-1280),0.02227654680609703,5.227924475548207e-07 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(10240-128),0.012178877368569374,3.466562858989164e-08 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_ff_net_2.lora_down.weight,(128-5120),0.011033281683921814,4.494391347975579e-08 lora_unet_up_blocks_1_attentions_0_transformer_blocks_0_ff_net_2.lora_up.weight,(1280-128),0.026162046939134598,4.78766764899774e-07 lora_unet_up_blocks_1_attentions_1_proj_in.lora_down.weight,(128-1280-1-1),0.022206783294677734,4.415646799316164e-07 lora_unet_up_blocks_1_attentions_1_proj_in.lora_up.weight,(1280-128-1-1),0.007817922160029411,1.7703929700019216e-07 lora_unet_up_blocks_1_attentions_1_proj_out.lora_down.weight,(128-1280-1-1),0.022300858050584793,1.1844922198633867e-07 lora_unet_up_blocks_1_attentions_1_proj_out.lora_up.weight,(1280-128-1-1),0.007528032176196575,3.747394217157307e-08 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-1280),0.022275911644101143,5.4563546569852406e-08 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn1_to_k.lora_up.weight,(1280-128),0.01406577043235302,3.914684612027486e-07 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-1280),0.02218853309750557,9.13481414954731e-08 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(1280-128),0.008929646573960781,6.251028850101648e-08 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-1280),0.022265704348683357,2.07257542683692e-07 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn1_to_q.lora_up.weight,(1280-128),0.01426897756755352,1.3116398633883364e-07 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-1280),0.022237936034798622,2.88015485239157e-07 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn1_to_v.lora_up.weight,(1280-128),0.009421738795936108,3.886792754315138e-08 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028699060901999474,1.4543410031819803e-08 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn2_to_k.lora_up.weight,(1280-128),0.013541599735617638,5.595376606493119e-08 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-1280),0.022296389564871788,4.311220180852615e-08 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(1280-128),0.011824141256511211,3.88823151453721e-09 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-1280),0.022289132699370384,1.0538372663404516e-07 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn2_to_q.lora_up.weight,(1280-128),0.016380954533815384,2.448333589200047e-07 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.02750064991414547,1.0823397360582021e-06 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_attn2_to_v.lora_up.weight,(1280-128),0.010105499997735023,1.0500959035653068e-07 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-1280),0.022256771102547646,5.037573487243208e-07 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(10240-128),0.01016799546778202,2.749373395616317e-09 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_ff_net_2.lora_down.weight,(128-5120),0.011050654575228691,5.552449611201382e-09 lora_unet_up_blocks_1_attentions_1_transformer_blocks_0_ff_net_2.lora_up.weight,(1280-128),0.02184833213686943,6.753146521987219e-07 lora_unet_up_blocks_1_attentions_2_proj_in.lora_down.weight,(128-1280-1-1),0.022156972438097,1.1219084328217832e-08 lora_unet_up_blocks_1_attentions_2_proj_in.lora_up.weight,(1280-128-1-1),0.006962059531360865,1.052547133895132e-07 lora_unet_up_blocks_1_attentions_2_proj_out.lora_down.weight,(128-1280-1-1),0.022273767739534378,1.5125934282878006e-07 lora_unet_up_blocks_1_attentions_2_proj_out.lora_up.weight,(1280-128-1-1),0.006835971027612686,1.546102446070563e-08 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-1280),0.02228235825896263,1.655696202362833e-08 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn1_to_k.lora_up.weight,(1280-128),0.014829076826572418,1.2054911735503993e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-1280),0.0217503122985363,4.674901177281754e-08 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(1280-128),0.00954731460660696,1.2442286845271155e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-1280),0.022291745990514755,3.152042040710512e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn1_to_q.lora_up.weight,(1280-128),0.014903649687767029,1.9145429064337804e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-1280),0.022231753915548325,3.384114961590967e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn1_to_v.lora_up.weight,(1280-128),0.0096674133092165,3.820205307647484e-08 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.02869029901921749,8.479614734824281e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_k.lora_up.weight,(1280-128),0.01443397719413042,9.231791153752056e-08 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-1280),0.02227676287293434,5.647094880600889e-08 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(1280-128),0.011246388778090477,7.649092736983221e-08 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-1280),0.022302594035863876,1.566059637525541e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_q.lora_up.weight,(1280-128),0.017705220729112625,4.354885447810375e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.026816343888640404,5.153858069206763e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_v.lora_up.weight,(1280-128),0.010233255103230476,4.241649520508872e-08 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-1280),0.022276366129517555,7.982220608937496e-07 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(10240-128),0.010309142991900444,4.132897579722794e-09 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_ff_net_2.lora_down.weight,(128-5120),0.011017205193638802,6.27247809248388e-09 lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_ff_net_2.lora_up.weight,(1280-128),0.02200053259730339,5.295219693834952e-07 lora_unet_up_blocks_1_resnets_0_conv1.lora_down.weight,(128-2560-3-3),0.005197310820221901,2.5437338346812055e-10 lora_unet_up_blocks_1_resnets_0_conv1.lora_up.weight,(1280-128-1-1),0.0375458225607872,2.0593159888449009e-07 lora_unet_up_blocks_1_resnets_0_conv2.lora_down.weight,(128-1280-3-3),0.007369490340352058,1.9698709330384645e-09 lora_unet_up_blocks_1_resnets_0_conv2.lora_up.weight,(1280-128-1-1),0.030164217576384544,2.2897265239407716e-07 lora_unet_up_blocks_1_resnets_0_conv_shortcut.lora_down.weight,(128-2560-1-1),0.013668830506503582,3.2389444015734625e-08 lora_unet_up_blocks_1_resnets_0_conv_shortcut.lora_up.weight,(1280-128-1-1),0.009220477193593979,1.8995152117895486e-07 lora_unet_up_blocks_1_resnets_0_time_emb_proj.lora_down.weight,(128-1280),0.020909015089273453,9.890181473792836e-08 lora_unet_up_blocks_1_resnets_0_time_emb_proj.lora_up.weight,(1280-128),0.0023382455110549927,3.954425764618463e-08 lora_unet_up_blocks_1_resnets_1_conv1.lora_down.weight,(128-2560-3-3),0.005226499401032925,2.662828846311527e-10 lora_unet_up_blocks_1_resnets_1_conv1.lora_up.weight,(1280-128-1-1),0.03534857556223869,1.3727368468607892e-06 lora_unet_up_blocks_1_resnets_1_conv2.lora_down.weight,(128-1280-3-3),0.007365470286458731,2.8881363967059315e-09 lora_unet_up_blocks_1_resnets_1_conv2.lora_up.weight,(1280-128-1-1),0.027655789628624916,1.6385243384320347e-07 lora_unet_up_blocks_1_resnets_1_conv_shortcut.lora_down.weight,(128-2560-1-1),0.014390352182090282,6.348277459267138e-10 lora_unet_up_blocks_1_resnets_1_conv_shortcut.lora_up.weight,(1280-128-1-1),0.007813962176442146,6.106741778921787e-08 lora_unet_up_blocks_1_resnets_1_time_emb_proj.lora_down.weight,(128-1280),0.02098895236849785,1.4903066869464965e-07 lora_unet_up_blocks_1_resnets_1_time_emb_proj.lora_up.weight,(1280-128),0.0022098522167652845,1.152987705310693e-10 lora_unet_up_blocks_1_resnets_2_conv1.lora_down.weight,(128-1920-3-3),0.006035279016941786,3.9503697535359095e-10 lora_unet_up_blocks_1_resnets_2_conv1.lora_up.weight,(1280-128-1-1),0.035121720284223557,2.341671034855608e-07 lora_unet_up_blocks_1_resnets_2_conv2.lora_down.weight,(128-1280-3-3),0.00729749258607626,2.0387133758958953e-08 lora_unet_up_blocks_1_resnets_2_conv2.lora_up.weight,(1280-128-1-1),0.03342372551560402,3.0697498232257203e-07 lora_unet_up_blocks_1_resnets_2_conv_shortcut.lora_down.weight,(128-1920-1-1),0.017319880425930023,1.984382436148735e-08 lora_unet_up_blocks_1_resnets_2_conv_shortcut.lora_up.weight,(1280-128-1-1),0.004719236865639687,2.1091697988140368e-07 lora_unet_up_blocks_1_resnets_2_time_emb_proj.lora_down.weight,(128-1280),0.02099677175283432,1.3231467477226033e-08 lora_unet_up_blocks_1_resnets_2_time_emb_proj.lora_up.weight,(1280-128),0.002381554339081049,1.2515964264991908e-08 lora_unet_up_blocks_1_upsamplers_0_conv.lora_down.weight,(128-1280-3-3),0.007389573846012354,2.868065251959706e-08 lora_unet_up_blocks_1_upsamplers_0_conv.lora_up.weight,(1280-128-1-1),0.017832715064287186,3.7903836869190854e-07 lora_unet_up_blocks_2_attentions_0_proj_in.lora_down.weight,(128-640-1-1),0.031482480466365814,1.785694223599421e-07 lora_unet_up_blocks_2_attentions_0_proj_in.lora_up.weight,(640-128-1-1),0.00737210875377059,3.121649427839657e-08 lora_unet_up_blocks_2_attentions_0_proj_out.lora_down.weight,(128-640-1-1),0.03149263188242912,3.6485209875536384e-07 lora_unet_up_blocks_2_attentions_0_proj_out.lora_up.weight,(640-128-1-1),0.010358509607613087,2.6543710873738746e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-640),0.03151784837245941,4.075161896821555e-08 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn1_to_k.lora_up.weight,(640-128),0.011853165924549103,3.194396640537889e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-640),0.03132614120841026,4.11574347936039e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(640-128),0.009338026866316795,3.767060618997675e-08 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-640),0.03151374310255051,4.0531321587877756e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn1_to_q.lora_up.weight,(640-128),0.012034586630761623,1.1803521715592069e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-640),0.03129061311483383,1.750915657794394e-06 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn1_to_v.lora_up.weight,(640-128),0.010274907574057579,1.4924162883289682e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028756892308592796,1.4798715710639954e-06 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn2_to_k.lora_up.weight,(640-128),0.01634342037141323,4.2352658624622563e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-640),0.03135021775960922,4.986784460925264e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(640-128),0.010437595658004284,2.2517575359870534e-08 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-640),0.031599193811416626,1.2155547324255167e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn2_to_q.lora_up.weight,(640-128),0.0164242722094059,2.745372569279425e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.02624264359474182,1.8044374883174896e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_attn2_to_v.lora_up.weight,(640-128),0.012555432505905628,1.831275966424073e-07 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-640),0.03150327503681183,3.356615479788161e-06 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(5120-128),0.009472614154219627,4.751527615809437e-09 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_ff_net_2.lora_down.weight,(128-2560),0.015361413359642029,1.1408701539039612e-08 lora_unet_up_blocks_2_attentions_0_transformer_blocks_0_ff_net_2.lora_up.weight,(640-128),0.02386390045285225,2.400521452727844e-07 lora_unet_up_blocks_2_attentions_1_proj_in.lora_down.weight,(128-640-1-1),0.03133314847946167,3.718022298926371e-07 lora_unet_up_blocks_2_attentions_1_proj_in.lora_up.weight,(640-128-1-1),0.006672989577054977,7.941785007403723e-09 lora_unet_up_blocks_2_attentions_1_proj_out.lora_down.weight,(128-640-1-1),0.03154124692082405,4.695401969456725e-07 lora_unet_up_blocks_2_attentions_1_proj_out.lora_up.weight,(640-128-1-1),0.008469762280583382,1.0906045844194523e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-640),0.031545333564281464,8.931225181640912e-08 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn1_to_k.lora_up.weight,(640-128),0.013217255473136902,1.404615623812333e-08 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-640),0.03113684058189392,8.578754204791039e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(640-128),0.008592324331402779,1.42045877282726e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-640),0.031495433300733566,6.051499212844647e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn1_to_q.lora_up.weight,(640-128),0.013562796637415886,5.371725251279713e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-640),0.031151633709669113,1.1358964684404782e-06 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn1_to_v.lora_up.weight,(640-128),0.009705127216875553,9.063646899676314e-08 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028777271509170532,7.152557373046875e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn2_to_k.lora_up.weight,(640-128),0.01573893427848816,1.2752384748182521e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-640),0.03137066215276718,1.0962889263055331e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(640-128),0.0078046200796961784,7.997337903020707e-09 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-640),0.03152909874916077,2.2402127797249705e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn2_to_q.lora_up.weight,(640-128),0.01616772450506687,4.540935378827271e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.026802124455571175,4.0745362639427185e-08 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_attn2_to_v.lora_up.weight,(640-128),0.011308247223496437,5.664907121172291e-08 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-640),0.03150882571935654,2.4709695480851224e-06 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(5120-128),0.009192240424454212,1.3072148696835484e-08 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_ff_net_2.lora_down.weight,(128-2560),0.015432961285114288,3.557652235031128e-07 lora_unet_up_blocks_2_attentions_1_transformer_blocks_0_ff_net_2.lora_up.weight,(640-128),0.022338639944791794,3.3370272944921453e-07 lora_unet_up_blocks_2_attentions_2_proj_in.lora_down.weight,(128-640-1-1),0.03139403089880943,8.139795113493165e-07 lora_unet_up_blocks_2_attentions_2_proj_in.lora_up.weight,(640-128-1-1),0.005615397356450558,1.035808860905263e-08 lora_unet_up_blocks_2_attentions_2_proj_out.lora_down.weight,(128-640-1-1),0.03157135099172592,5.020336857342045e-07 lora_unet_up_blocks_2_attentions_2_proj_out.lora_up.weight,(640-128-1-1),0.006972128059715033,8.319772604181708e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-640),0.03142239898443222,1.0205663159013056e-07 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn1_to_k.lora_up.weight,(640-128),0.011869070120155811,3.461720510244959e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-640),0.030899718403816223,7.466365303798739e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(640-128),0.008088083937764168,6.596796708890906e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-640),0.03147032856941223,3.134748851607583e-07 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn1_to_q.lora_up.weight,(640-128),0.012790965847671032,4.503964845525843e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-640),0.03115309402346611,1.8035368753999137e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn1_to_v.lora_up.weight,(640-128),0.008387573063373566,8.755554148365263e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028765538707375526,2.7939677238464355e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn2_to_k.lora_up.weight,(640-128),0.015567811205983162,1.0610811784772523e-07 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-640),0.031383953988552094,1.0007483979279641e-06 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(640-128),0.006153458263725042,1.0925091942226572e-07 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-640),0.03150609880685806,2.314283619853086e-06 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn2_to_q.lora_up.weight,(640-128),0.016442108899354935,3.572041862298647e-07 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.026463111862540245,2.4586915969848633e-07 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_attn2_to_v.lora_up.weight,(640-128),0.010104911401867867,3.582430849746743e-07 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-640),0.03140022233128548,2.2467580151896982e-07 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(5120-128),0.008904803544282913,3.4919175107006595e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_ff_net_2.lora_down.weight,(128-2560),0.015341445803642273,8.66129994392395e-08 lora_unet_up_blocks_2_attentions_2_transformer_blocks_0_ff_net_2.lora_up.weight,(640-128),0.021548409014940262,8.745954573896597e-07 lora_unet_up_blocks_2_resnets_0_conv1.lora_down.weight,(128-1920-3-3),0.0059505547396838665,2.2087132123260744e-09 lora_unet_up_blocks_2_resnets_0_conv1.lora_up.weight,(640-128-1-1),0.04317503049969673,5.563288709709013e-07 lora_unet_up_blocks_2_resnets_0_conv2.lora_down.weight,(128-640-3-3),0.010173672810196877,1.0244548320770264e-08 lora_unet_up_blocks_2_resnets_0_conv2.lora_up.weight,(640-128-1-1),0.029525870457291603,7.350826081164996e-08 lora_unet_up_blocks_2_resnets_0_conv_shortcut.lora_down.weight,(128-1920-1-1),0.012765919789671898,7.12461769580841e-08 lora_unet_up_blocks_2_resnets_0_conv_shortcut.lora_up.weight,(640-128-1-1),0.016201287508010864,2.757732318059425e-07 lora_unet_up_blocks_2_resnets_0_time_emb_proj.lora_down.weight,(128-1280),0.0213811956346035,1.9953586161136627e-07 lora_unet_up_blocks_2_resnets_0_time_emb_proj.lora_up.weight,(640-128),0.0033218779135495424,4.996178581961885e-09 lora_unet_up_blocks_2_resnets_1_conv1.lora_down.weight,(128-1280-3-3),0.0072383577935397625,3.530556469399926e-08 lora_unet_up_blocks_2_resnets_1_conv1.lora_up.weight,(640-128-1-1),0.03221921622753143,7.882579780016385e-07 lora_unet_up_blocks_2_resnets_1_conv2.lora_down.weight,(128-640-3-3),0.00995341781526804,7.05040292814374e-09 lora_unet_up_blocks_2_resnets_1_conv2.lora_up.weight,(640-128-1-1),0.02493247576057911,2.726679326769954e-08 lora_unet_up_blocks_2_resnets_1_conv_shortcut.lora_down.weight,(128-1280-1-1),0.018691226840019226,3.2014213502407074e-08 lora_unet_up_blocks_2_resnets_1_conv_shortcut.lora_up.weight,(640-128-1-1),0.009205872192978859,1.011097872094524e-08 lora_unet_up_blocks_2_resnets_1_time_emb_proj.lora_down.weight,(128-1280),0.021661842241883278,1.62515789270401e-07 lora_unet_up_blocks_2_resnets_1_time_emb_proj.lora_up.weight,(640-128),0.004613420460373163,1.1543129829760801e-07 lora_unet_up_blocks_2_resnets_2_conv1.lora_down.weight,(128-960-3-3),0.00837259367108345,2.3283064365386963e-09 lora_unet_up_blocks_2_resnets_2_conv1.lora_up.weight,(640-128-1-1),0.028894493356347084,5.55317178907444e-08 lora_unet_up_blocks_2_resnets_2_conv2.lora_down.weight,(128-640-3-3),0.009784400463104248,2.9336661100387573e-08 lora_unet_up_blocks_2_resnets_2_conv2.lora_up.weight,(640-128-1-1),0.02545870840549469,2.517060124773707e-07 lora_unet_up_blocks_2_resnets_2_conv_shortcut.lora_down.weight,(128-960-1-1),0.024969432502985,1.993030309677124e-07 lora_unet_up_blocks_2_resnets_2_conv_shortcut.lora_up.weight,(640-128-1-1),0.003912205807864666,4.807630205050373e-08 lora_unet_up_blocks_2_resnets_2_time_emb_proj.lora_down.weight,(128-1280),0.02166583202779293,3.5390257835388184e-08 lora_unet_up_blocks_2_resnets_2_time_emb_proj.lora_up.weight,(640-128),0.00483495369553566,3.1460995586485296e-08 lora_unet_up_blocks_2_upsamplers_0_conv.lora_down.weight,(128-640-3-3),0.010398396290838718,1.1641532182693481e-08 lora_unet_up_blocks_2_upsamplers_0_conv.lora_up.weight,(640-128-1-1),0.014509675092995167,8.243486604442296e-07 lora_unet_up_blocks_3_attentions_0_proj_in.lora_down.weight,(128-320-1-1),0.04452945664525032,1.9452593278401764e-06 lora_unet_up_blocks_3_attentions_0_proj_in.lora_up.weight,(320-128-1-1),0.005616842769086361,8.65432259189447e-09 lora_unet_up_blocks_3_attentions_0_proj_out.lora_down.weight,(128-320-1-1),0.04456087946891785,2.392194119238411e-06 lora_unet_up_blocks_3_attentions_0_proj_out.lora_up.weight,(320-128-1-1),0.008974398486316204,8.568369480599358e-08 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-320),0.04432270675897598,4.527545058863325e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn1_to_k.lora_up.weight,(320-128),0.010653076693415642,2.0965320857158076e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-320),0.04448689520359039,1.7149553741546697e-06 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(320-128),0.007018988020718098,3.8564547821806627e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-320),0.044529661536216736,2.332526719328598e-06 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn1_to_q.lora_up.weight,(320-128),0.01189712155610323,9.926552024808188e-09 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-320),0.044114310294389725,3.2738844879531825e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn1_to_v.lora_up.weight,(320-128),0.007393618114292622,1.9764540581945766e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028732873499393463,7.189810276031494e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn2_to_k.lora_up.weight,(320-128),0.015247011557221413,7.695618933212245e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-320),0.043973930180072784,1.0991444696628605e-06 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(320-128),0.003322028089314699,1.867796051158166e-08 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-320),0.04454230144619942,3.8304054328364145e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn2_to_q.lora_up.weight,(320-128),0.012469004839658737,1.4940164305699e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.028225481510162354,1.4994293451309204e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_attn2_to_v.lora_up.weight,(320-128),0.007692402694374323,2.9525557465603924e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-320),0.044464677572250366,7.701404456383898e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(2560-128),0.007301993668079376,9.113118437653611e-08 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_ff_net_2.lora_down.weight,(128-1280),0.022050676867365837,5.103647708892822e-07 lora_unet_up_blocks_3_attentions_0_transformer_blocks_0_ff_net_2.lora_up.weight,(320-128),0.017579715698957443,1.1007051625711028e-07 lora_unet_up_blocks_3_attentions_1_proj_in.lora_down.weight,(128-320-1-1),0.04450801759958267,8.715419426152948e-07 lora_unet_up_blocks_3_attentions_1_proj_in.lora_up.weight,(320-128-1-1),0.004715863615274429,2.2250972619985987e-08 lora_unet_up_blocks_3_attentions_1_proj_out.lora_down.weight,(128-320-1-1),0.04441447928547859,5.905449143028818e-07 lora_unet_up_blocks_3_attentions_1_proj_out.lora_up.weight,(320-128-1-1),0.007593308575451374,1.9023613617719093e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-320),0.04443392530083656,2.709331283767824e-06 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn1_to_k.lora_up.weight,(320-128),0.011607426218688488,1.4625696849179803e-06 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-320),0.043690573424100876,3.0564729058824014e-06 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(320-128),0.006496323738247156,4.00115965248915e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-320),0.0444464385509491,2.5227523110515904e-06 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn1_to_q.lora_up.weight,(320-128),0.012978890910744667,4.856497071159538e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-320),0.04374261572957039,1.1197621496705779e-08 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn1_to_v.lora_up.weight,(320-128),0.006678114179521799,1.4377428669831716e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.028750531375408173,7.078051567077637e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn2_to_k.lora_up.weight,(320-128),0.016766533255577087,1.93098981071671e-06 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-320),0.04371967911720276,1.0041791256298893e-06 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(320-128),0.0029893144965171814,1.3704783974333168e-08 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-320),0.04443332925438881,3.1372044873023697e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn2_to_q.lora_up.weight,(320-128),0.013420340605080128,6.501210947362779e-08 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.027652869001030922,1.3597309589385986e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_attn2_to_v.lora_up.weight,(320-128),0.008508380502462387,9.007022754303762e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-320),0.044263649731874466,4.310423378228734e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(2560-128),0.006486666854470968,3.824291230913879e-11 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_ff_net_2.lora_down.weight,(128-1280),0.021998200565576553,4.3958425521850586e-07 lora_unet_up_blocks_3_attentions_1_transformer_blocks_0_ff_net_2.lora_up.weight,(320-128),0.016004586592316628,2.3361890555406717e-07 lora_unet_up_blocks_3_attentions_2_proj_in.lora_down.weight,(128-320-1-1),0.04412155598402023,7.015417509137478e-07 lora_unet_up_blocks_3_attentions_2_proj_in.lora_up.weight,(320-128-1-1),0.004278108477592468,9.062659955816343e-08 lora_unet_up_blocks_3_attentions_2_proj_out.lora_down.weight,(128-320-1-1),0.04427538067102432,8.542396017219289e-07 lora_unet_up_blocks_3_attentions_2_proj_out.lora_up.weight,(320-128-1-1),0.007343068718910217,1.333467025688151e-07 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn1_to_k.lora_down.weight,(128-320),0.04431452602148056,1.4835496813248028e-06 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn1_to_k.lora_up.weight,(320-128),0.01205441914498806,4.1441865050728666e-08 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn1_to_out_0.lora_down.weight,(128-320),0.0436394102871418,6.232166356312518e-07 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn1_to_out_0.lora_up.weight,(320-128),0.007266148924827576,7.480742283405561e-07 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn1_to_q.lora_down.weight,(128-320),0.044493772089481354,3.193056272721151e-06 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn1_to_q.lora_up.weight,(320-128),0.01501234620809555,3.240506885049399e-07 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn1_to_v.lora_down.weight,(128-320),0.043818436563014984,1.6687563402228989e-06 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn1_to_v.lora_up.weight,(320-128),0.006822054274380207,8.650536642562656e-08 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn2_to_k.lora_down.weight,(128-768),0.02869974635541439,1.0635703802108765e-06 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn2_to_k.lora_up.weight,(320-128),0.01349611859768629,5.011053758607886e-07 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn2_to_out_0.lora_down.weight,(128-320),0.043756984174251556,2.0688660242740298e-06 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn2_to_out_0.lora_up.weight,(320-128),0.0023334103170782328,2.443455002776318e-07 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn2_to_q.lora_down.weight,(128-320),0.044480737298727036,1.2367737554086489e-06 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn2_to_q.lora_up.weight,(320-128),0.011029778979718685,2.3525867476337226e-08 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn2_to_v.lora_down.weight,(128-768),0.027012793347239494,5.159527063369751e-07 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_attn2_to_v.lora_up.weight,(320-128),0.007222442887723446,2.0431940583875985e-07 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_ff_net_0_proj.lora_down.weight,(128-320),0.04431402310729027,1.2588342315211776e-06 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_ff_net_0_proj.lora_up.weight,(2560-128),0.007004083134233952,5.591475371602428e-08 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_ff_net_2.lora_down.weight,(128-1280),0.02197149023413658,7.59027898311615e-08 lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_ff_net_2.lora_up.weight,(320-128),0.018417323008179665,2.2934293610887835e-06 lora_unet_up_blocks_3_resnets_0_conv1.lora_down.weight,(128-960-3-3),0.00809300784021616,2.722639669272553e-09 lora_unet_up_blocks_3_resnets_0_conv1.lora_up.weight,(320-128-1-1),0.034627515822649,9.29243697100901e-07 lora_unet_up_blocks_3_resnets_0_conv2.lora_down.weight,(128-320-3-3),0.014146695844829082,6.193295121192932e-08 lora_unet_up_blocks_3_resnets_0_conv2.lora_up.weight,(320-128-1-1),0.02037164941430092,1.2360909806830023e-07 lora_unet_up_blocks_3_resnets_0_conv_shortcut.lora_down.weight,(128-960-1-1),0.02183082513511181,1.9604340195655823e-07 lora_unet_up_blocks_3_resnets_0_conv_shortcut.lora_up.weight,(320-128-1-1),0.008654634468257427,2.0569737557707413e-07 lora_unet_up_blocks_3_resnets_0_time_emb_proj.lora_down.weight,(128-1280),0.021876981481909752,3.306195139884949e-08 lora_unet_up_blocks_3_resnets_0_time_emb_proj.lora_up.weight,(320-128),0.0056814090348780155,7.853545724856303e-08 lora_unet_up_blocks_3_resnets_1_conv1.lora_down.weight,(128-640-3-3),0.010024172253906727,5.750734999310225e-09 lora_unet_up_blocks_3_resnets_1_conv1.lora_up.weight,(320-128-1-1),0.025099435821175575,3.801945069881185e-07 lora_unet_up_blocks_3_resnets_1_conv2.lora_down.weight,(128-320-3-3),0.014083118177950382,2.7939677238464355e-09 lora_unet_up_blocks_3_resnets_1_conv2.lora_up.weight,(320-128-1-1),0.015840066596865654,8.598321414865495e-07 lora_unet_up_blocks_3_resnets_1_conv_shortcut.lora_down.weight,(128-640-1-1),0.028250956907868385,1.5212572179734707e-07 lora_unet_up_blocks_3_resnets_1_conv_shortcut.lora_up.weight,(320-128-1-1),0.006096608005464077,8.822191688295788e-08 lora_unet_up_blocks_3_resnets_1_time_emb_proj.lora_down.weight,(128-1280),0.021758228540420532,4.163011908531189e-07 lora_unet_up_blocks_3_resnets_1_time_emb_proj.lora_up.weight,(320-128),0.0046862391754984856,7.647699362678395e-08 lora_unet_up_blocks_3_resnets_2_conv1.lora_down.weight,(128-640-3-3),0.009737825021147728,5.587935447692871e-09 lora_unet_up_blocks_3_resnets_2_conv1.lora_up.weight,(320-128-1-1),0.02118118666112423,1.1859781068324082e-07 lora_unet_up_blocks_3_resnets_2_conv2.lora_down.weight,(128-320-3-3),0.014352843165397644,1.2968666851520538e-07 lora_unet_up_blocks_3_resnets_2_conv2.lora_up.weight,(320-128-1-1),0.015870559960603714,3.3438683999520435e-07 lora_unet_up_blocks_3_resnets_2_conv_shortcut.lora_down.weight,(128-640-1-1),0.029842844232916832,2.6961788535118103e-07 lora_unet_up_blocks_3_resnets_2_conv_shortcut.lora_up.weight,(320-128-1-1),0.00460923183709383,8.782388505323979e-08 lora_unet_up_blocks_3_resnets_2_time_emb_proj.lora_down.weight,(128-1280),0.02168363705277443,2.100132405757904e-07 lora_unet_up_blocks_3_resnets_2_time_emb_proj.lora_up.weight,(320-128),0.005326862446963787,9.600677941534741e-08

alenknight commented 1 year ago

sorry, gonna need some assistance... how do I run the networks/check_lora_weights.py file? (I found it... just run in pycharm? ) also.... not sure where to run --help. I thought it'd be a file in the directory...

meanwhile homely DreamFilmVFX's details maybe will help better.

kohya-ss commented 1 year ago

Thank you sharing the details for weights.

It has values that appear to be valid, so it should have some effect. If you increase the weight (multiplier) when applying it, do you see any change in the generated image?

alenknight commented 1 year ago

Thank you sharing the details for weights.

It has values that appear to be valid, so it should have some effect. If you increase the weight (multiplier) when applying it, do you see any change in the generated image?

how do we check the weights. I'd like to do this on mine as well to confirm.... but also I wasn't sure what to do when you mentioned --help. typing it into cmd terminal in windows didn't seem to do anything pertaining to kohya

alenknight commented 1 year ago

interesting. now it's working. seems it was an A4 issue. I even ran the LORA I extracted when I opened this ticket... and it's having an effect.
still.... if you could help me understand how to check the weights or how to run the --help you mentioned so I can check weights for myself in the future; would be helpful.

thanks!

kohya-ss commented 11 months ago

I'm happy to hear it's working.

From the command prompt, please run python networks\check_lora_weights.py path_to_lora.safetensors to show the weights of LoRA.