search

OpenGVLab / Vision-RWKV

Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures
https://arxiv.org/abs/2403.02308
Apache License 2.0
371 stars 14 forks source link

Is RWKV6 unidrectional or bidrectional? #17

Closed Yaziwel closed 6 months ago

Yaziwel commented 6 months ago

Have you incorporated a bidirectional attention mechanism into RWKV6, or are you adhering to the unidirectional attention mechanism described in the original paper?

duanduanduanyuchen commented 6 months ago

Thanks for your attention! VRWKV6 is also bidirectional attention like VRWKV.