search

vgaraujov / CPC-NLP-PyTorch

Implementation of Contrastive Predictive Coding for Natural Language
10 stars 3 forks source link

hi ， a question about multi-gpu #1

Closed jun0wanan closed 3 years ago

jun0wanan commented 3 years ago

Will this make the result bad ?

vgaraujov commented 3 years ago

Hello

It shouldn't, the multiple gpu option allows you to use more batches during training

jun0wanan commented 3 years ago

Hello

It shouldn't, the multiple gpu option allows you to use more batches during training

trained on 8 GPUs, each with a batch size of 64...

Hi , I want to run a batch size of 64 on 8gpus ,but not each with a batch size of 64, can I ? Thank you

vgaraujov commented 3 years ago

If I get it right, you want to use 8 GPUs with a batch size of 8 each. In that case, you need to set the batch_size parameter to 8 in the config_cpc.yaml file.

With that configuration, the model will have a different result because it will receive fewer negative samples.