It doesn't follow OOP principle to check layers by name instead of type by inheritence. Also naming is in quite convenient, e.g. when I just want to name the module self.query instead of self.lora_query so that I can load the original weights.
Also changed it to set lora layers requires_grad=True, as the function name suggests (requires_grad can default to false).
It doesn't follow OOP principle to check layers by name instead of type by inheritence. Also naming is in quite convenient, e.g. when I just want to name the module self.query instead of self.lora_query so that I can load the original weights. Also changed it to set lora layers requires_grad=True, as the function name suggests (requires_grad can default to false).