search

Langboat / mengzi-retrieval-lm

An experimental implementation of the retrieval-enhanced language model
Apache License 2.0
75 stars 5 forks source link

About the compute resources #20

Open WorldHellooo opened 1 year ago

WorldHellooo commented 1 year ago

Thanks for making your work public! Want to know how many computing resources were used for training and retrieval when you train the GPT-125M model?

daniellefisla commented 1 year ago

I am curious as well.

Ag2S1 commented 1 year ago

We used two 8*A100 40G servers for training, one as an index server and one as a training server. If you want to train a larger model, 1 index server can also be used, just increase the number of training servers