search

OpenNMT / CTranslate2

Fast inference engine for Transformer models
https://opennmt.net/CTranslate2
MIT License
3.21k stars 282 forks source link

sqlcoder (7b) from mistral model #1591

Closed JunHyungKang closed 8 months ago

JunHyungKang commented 8 months ago

during converting sqlcoder(7b), following errors are occured. As I know, sqlcoder is fine-tuned from mistral which is supported by CT2.

!ct2-transformers-converter --model defog/sqlcoder-7b --quantization int8_bfloat16 --output_dir ./sqlcoder-7b_int8_bfloat16_ct2 --force --trust_remote_code

!ct2-transformers-converter --model defog/sqlcoder-7b --quantization int8_bfloat16 --output_dir ./sqlcoder-7b_int8_bfloat16_ct2 --force --trust_remote_code

vince62s commented 8 months ago

you posted a command line not an error.