search

xfactlab / orpo

Official repository for ORPO
Apache License 2.0
421 stars 39 forks source link

Update main.py #24

Open ganeshkrishnan1 opened 6 months ago

ganeshkrishnan1 commented 6 months ago

bfloat16 is not supported on ampere devices so if flash attention 2 is not supported its an ampere device and dtype has to be float16