Does unilm/e5 support deepspeed or other accelerator?

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

https://aka.ms/GeneralAI

MIT License

20.2k stars 2.55k forks source link

Does unilm/e5 support deepspeed or other accelerator? #1424

Open mengyao00 opened 10 months ago

mengyao00 commented 10 months ago

Describe Model I am using (UniLM, MiniLM, LayoutLM ...):e5 Can I use deepspeed for evaluating beir benchmark? Or does it support other strategies? Can I use fp16? Would it impact results?

intfloat commented 10 months ago

Hi @mengyao00 ,

Can I use deepspeed for evaluating beir benchmark?

Currently we do not support using DeepSpeed for inference. I think the main advantage of DeepSpeed is about reducing training GPU memory usage. For inference, traditional data parallel is enough.

Can I use fp16? Would it impact results?

Yes, in fact, our evaluation results are all based on fp16.