search

OpenGVLab / Vision-RWKV

Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures
https://arxiv.org/abs/2403.02308
Apache License 2.0
371 stars 14 forks source link

VRWKV6 fp16? Mixed Precision Training? #30

Open FlotingDream opened 4 months ago

FlotingDream commented 4 months ago

VRWKV6 fp16? Mixed Precision Training?

duanduanduanyuchen commented 4 months ago

Hi, currently we transferred all types of data to fp32 in vrwkv like this: link. You can use this simple way for vrwkv6 as well.