search

davidtellez / contrastive-predictive-coding

Keras implementation of Representation Learning with Contrastive Predictive Coding
525 stars 120 forks source link

I can reproduce the repo's result by reduce the positive samples. #17

Open 255-1 opened 2 years ago

255-1 commented 2 years ago

I try many times to repeat the result, but my binary_accuracy always 50%, it's horrible. then i find that an issue mention that the positive_sample is the half of batch_size. I know that the number of negative samples are important to make NCELoss approximate the max-likelyhood estimation.so I change the train_model function of the "train_model.py".set positive_samples=1,

train_model

train_data = SortedNumberGenerator(batch_size=batch_size, subset='train', terms=terms, positive_samples=1, predict_terms=predict_terms, image_size=image_size, color=color, rescale=True) validation_data = SortedNumberGenerator(batch_size=batch_size, subset='valid', terms=terms, positive_samples=1, predict_terms=predict_terms, image_size=image_size, color=color, rescale=True)

then I got 96% binary_accuracy at the first epoch. Yep!