Does this really works?

[guivr]

Hi, I’ve tried running the ZipLoRA training twice and the style model seems to be not applied on the validation pictures I’m getting. I’ve tried with 2 different styles models. I can only see the subject model works, but then the the style is never applied to the subject even after 500 steps.

am I doing something wrong?

[practiceC]

I got the same result

[euminds]

Good results (not well) style： zipLora+ text prompt: a sbu dog in szn style bad results style ziplora+ text prompt:: a sbu dog in szn style

I think it doesn't work

[mkshing]

Thank you all for testing! In fact, I needed to tune the initial lora weights to get better results. https://twitter.com/mk1stats/status/1728037991795560573 But, if that tuning is needed, I could find no benefit of ZipLoRA. So, I doubt that there are some tips around training or there might be some bugs in my implementation. Now, I'm contacting the authors to ask for reviews. Ofc, I'm happy to get feedback from you if you find anything. Hopefully, my implementation works soon! Stay tuned :)

[KoTion]

I also encountered this problem, has anyone solved it? Additionally, interestingly, when testing content and style independently, the results were satisfactory. such as：use the trained ziplora, the prompt is “a sbu cat” and "a bicycle in szn style".

[eezywu]

same problem

[Saddss]

I also encountered this problem, has anyone solved it? Additionally, interestingly, when testing content and style independently, the results were satisfactory. such as：use the trained ziplora, the prompt is “a sbu cat” and "a bicycle in szn style".

Sorry to ask irrelevant questions here. KeyError occurs when I try to merge two loras with train_dreamtooth_ziploora_sdxl.py, do you know how to solve it? The error is follows:

Traceback (most recent call last)File "train_dreambooth_ziplora _sdxl.py", line 1817, in <module>main(args)
File "train_dreambooth_ziplora _sdxl.py”: line1019，in main merged_lora_weights_dict= merge_lora_weights(lora_weights, attn_name)
File "/root/autodl-tmp/ziplora-pytorch/ziplora_pytorch/utils.py", line 57, in merge_lora_ weights merged_weight = tensors[up_key] @ tensors[down_key]KeyError:'uet,wet. dow blocks.1,attentions.0.transformer blocks.0. attn1,to g. lora. up. weight’

[eezywu]

Thank you all for testing! In fact, I needed to tune the initial lora weights to get better results. https://twitter.com/mk1stats/status/1728037991795560573 But, if that tuning is needed, I could find no benefit of ZipLoRA. So, I doubt that there are some tips around training or there might be some bugs in my implementation. Now, I'm contacting the authors to ask for reviews. Ofc, I'm happy to get feedback from you if you find anything. Hopefully, my implementation works soon! Stay tuned :)

@mkshing hi, did you find any useful tips?