Open Double-bear opened 1 month ago
Yes it can be used to train that model; You're running OOM which you can solve via
per_device_train_batch_size
passage_max_len
--gradient_checkpointing
--split_emb
Consider taking a look at https://github.com/ContextualAI/gritlm/blob/main/scripts/training/train_gritlm_7b.sh & https://github.com/ContextualAI/gritlm/blob/main/scripts/training/train_gritlm_8x7b.sh
Hello. I wanted to use gritlm to a open-source embedding model —— gte-qwen2-7b-instruct, but I encountered some problems:
My GPU is A800, 80G, and I used 8 * A800. My submit script is following:
How can I solve this problem?