search

staoxiao / RetroMAE

Codebase for RetroMAE and beyond.
Apache License 2.0
228 stars 16 forks source link

How long is the pretraining of retromae with enhanced decoding #2

Closed ma787639046 closed 1 year ago

ma787639046 commented 1 year ago

Hi, thanks for your excellent work.

As described in the Experiment Settings Section, the pretraining of retromae with enhanced decoding was finished on 8*A100 GPUs. Could you please tell us how many hours you took to complete the pretraining？

Thanks.

staoxiao commented 1 year ago

Hi, it takes about 2 days on wiki and bookcorpus dataset, and no more than 1 day on msmarco dataset.

ma787639046 commented 1 year ago

Thanks a lot!