How long does inference take?

microsoft / ANCE

A novel embedding training algorithm leveraging ANN search and achieved SOTA retrieval on Trec DL 2019 and OpenQA benchmarks

MIT License

359 stars 50 forks source link

How long does inference take? #12

Open CosimoRulli opened 3 years ago

CosimoRulli commented 3 years ago

Hello developers, I followed the guidelines in your ReadMe to generate the dense representations for MS Marco Document Ranking, using the MaxP checkpoint that you provide. My process has been running for more than 80 hours, on a server with a T4 Tesla GPU and Intel Xeon Platinum CPU (looking at htop, I observe that it is running with a single thread). Is such a long inference time normal? Am I missing something to speedup this process?

juyongjiang commented 2 years ago

Hi, do you find the BM25 index generation?