Shape mismatch in paper?

[ro1406]

Hi, thanks for your work!

I was just going through the paper, and noticed a possible shape mismatch in the formulas, and wanted to clarify. In Equations 1 & 2, the product shown is BAx and BRAx. However, based on the formulation using SVD, i got the shapes of B, R, and A as follows:

• B is rxn
• R is rxr
• A is mxr

Hence, computing BRAx would produce a possible shape mismatch (BRA -> rxn X rxr X mxr shapes dont match).

Is this a typo and the original formula was meant to be ARBx?

Thanks and best of luck

[MohammadrezaBanaei]

Hi! Thank you for your interest in our work! We have indeed noticed a typo in the paper's formula, and we will correct it in the paper soon. Below is the version consistent with our code and experiments:

• Equation 1 should be: x(W + ∆W) = x(W + AB)
• Equation 2 should be: x W + x A R B Matrix Sizes:
  • W: m×n
  • SVD(W, r)=USV^T
  • U: m×r
  • S: r×r
  • V^T: r×n
• Lora_A: m × r
• Lora_B: r × n

Thanks again for catching this!