Question about sampling the x' (negative samples) & code.

rdevon / DIM

Deep InfoMax (DIM), or "Learning Deep Representations by Mutual Information Estimation and Maximization"

BSD 3-Clause "New" or "Revised" License

806 stars 102 forks source link

Question about sampling the x' (negative samples) & code. #39

Closed WOWNICE closed 4 years ago

WOWNICE commented 4 years ago

Hi,

I've read your paper, which is quite inspiring! But here I have some questions regarding the eq.4, which states "x' is an input sampled from \tilde{P} x P". Does it mean that we randomly pick (x,y) independently from the X and Y(Y is the generated representation distribution of X)? If so, which part of the code implements this sampling method?

Thanks a lot!

rdevon commented 4 years ago

x and x' are just sampled independently of each other, so for instance two different images in a batch where there are no dependencies on how different samples in the batch were sampled.

rdevon commented 4 years ago

meaning x' and x (together) are sampled from product of marginals P x P_tilde. P_tilde is just a placeholder for a "copy" of P

WOWNICE commented 4 years ago

Ahhh! Got it!

WOWNICE commented 4 years ago

Thanks for your kind reply!