In addition to adding unit tests I have made the following changes.
In LoRADense, replaced *args in __init__ with an explicit list of parameters. The public TF API is fairly stable and for code inspection it is easier if the PyCharm tooltip shows you the list of parameters.
In LoRADense, renamed merge_lora_weights() and unmerge_lora_weights() to merge_weights() and unmerge_weights(). I don't think it is necessary to repeat lora here. If possible and within reason, simpler names are preferable.
In LoRADense.call(), moved the if self.merged: check to the very beginning of the function, before casting happens. In the merged case we hand everything to the base class.
Fixed the LORA_WEIGHT_NAMES list in factory.py.
Added a convert_to_lora_layer helper function to make creating LoRA layers easier.
In addition to adding unit tests I have made the following changes.
LoRADense, replaced*argsin__init__with an explicit list of parameters. The public TF API is fairly stable and for code inspection it is easier if the PyCharm tooltip shows you the list of parameters.LoRADense, renamedmerge_lora_weights()andunmerge_lora_weights()tomerge_weights()andunmerge_weights(). I don't think it is necessary to repeatlorahere. If possible and within reason, simpler names are preferable.LoRADense.call(), moved theif self.merged:check to the very beginning of the function, before casting happens. In the merged case we hand everything to the base class.LORA_WEIGHT_NAMESlist infactory.py.convert_to_lora_layerhelper function to make creating LoRA layers easier.