It is an impressive extension of WKV from unidirectional to bidirectional. However, the current WKV is still limited to 1D sequences. I believe the relative bias |t-i|/T can be extended to (|x-i|/H + |y-j|/W), where x and y represent two directions in an image. It appears that this approach can also be transformed into a recurrent form with linear complexity.
It is an impressive extension of WKV from unidirectional to bidirectional. However, the current WKV is still limited to 1D sequences. I believe the relative bias |t-i|/T can be extended to (|x-i|/H + |y-j|/W), where x and y represent two directions in an image. It appears that this approach can also be transformed into a recurrent form with linear complexity.