search

Ronsor / nanoRWKV-bitnet

nanoRWKV + BitNet b1.58 (WIP)
MIT License
2 stars 0 forks source link

Any difference with performance between NanoRWKV-BitNET and NanoRWKV #1

Open NLPV2011 opened 3 months ago

Ronsor commented 3 months ago

image

Yes, at least at small model sizes, BitNet b158 performs worse than normal bfloat16 precision.