search

bofenghuang / vigogne

French instruction-following and chat models
Apache License 2.0
497 stars 47 forks source link

Update train_sft.py #19

Closed ell-hol closed 1 year ago

ell-hol commented 1 year ago

[Refactor] FutureWarning: prepare_model_for_int8_training is deprecated and will be removed in a future version. Use prepare_model_for_kbit_training instead

dedif1 commented 1 year ago

how can i give vigogne acces to internet

bofenghuang commented 1 year ago

Hi @ell-hol ,

Sorry for my late response. I was on vacation. Would it be possible for you to reopen the PR?

ell-hol commented 1 year ago

@bofenghuang Thank you for the follow up, I opened a new pull request. Would you like me to work on added support for bfloat16, 4bit Qlora with peft both in inference and training ? I have a working fork and would be happy to oblige.

bofenghuang commented 1 year ago

Hi @ell-hol,

Thank you for your PR.

These enhancements sound promising. Can't wait to review and learn from you!