How to use accelerate to multi gpu train and encode?

SeanLee97 / AnglE

Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard

https://arxiv.org/abs/2309.12871

MIT License

397 stars 30 forks source link

Closed WanBenLe closed 4 months ago

WanBenLe commented 4 months ago

It seems to be will rasie error..

WanBenLe commented 4 months ago

File "/AnglE-main/angle_emb/angle.py", line 459, in call input_ids = feature['input_ids'] KeyError: 'input_ids'

SeanLee97 commented 4 months ago

hi @WanBenLe , sorry for the delay. I tested training using multiGPU, it works fine.

Could you enclose your training script here? So that I can reproduce your issue.