NCE should be calculated for each output of the GRU (and future positions) instead of only a random index

jefflai108 / Contrastive-Predictive-Coding-PyTorch

Contrastive Predictive Coding for Automatic Speaker Verification

MIT License

472 stars 96 forks source link

NCE should be calculated for each output of the GRU (and future positions) instead of only a random index #21

Open Ieremie opened 2 years ago

Ieremie commented 2 years ago

In the implementation, a random position of the sequence is used to compute the NCE loss. The paper mentions that the GRU output at each step is used to predict 12 timesteps in the future.

"The output of the GRU at every timestep is used as the context c from which we predict 12 timesteps in the future using the contrastive loss"

Was this decision made to reduce training time?