search

IlyaGusev / rulm

Language modeling and instruction tuning for Russian
Apache License 2.0
455 stars 50 forks source link

convert_to_native.py 70b support #39

Open Displacer opened 8 months ago

Displacer commented 8 months ago

Is there reasons for not supporting 70b converting?

Are n_heads = 64, dim = 8192 for LLaMa v2 70b correct values?