search

HobbitLong / PyContrast

PyTorch implementation of Contrastive Learning methods
1.94k stars 186 forks source link

Proposition A.1 #2

Closed liun-online closed 4 years ago

liun-online commented 4 years ago

thanks your paper, and I think it's a fundamental job! But, I'm not clear that why I(y; x) = I(y; v1, v2) as given in proof in Proposition A.1. Could you please show me the reliable reason for it? Maybe, it's a naive question. I'm hope for your reply! Best wishes!

HobbitLong commented 4 years ago

Hi, @liun-online,

Thanks for your kind word. (1) I(y; x) >= I(y; v1, v2), as v1, v2 are functions of x, so we have this by data processing inequality; (2) I(y; v1, v2) >= I(y; v1) = I(y; x), as assumption. So it's equal.

Let me know if I am wrong or if you have further questions. I will close it by now but feel free to reopen.

HobbitLong commented 4 years ago

Sorry I feel like I should have a separate repo for "infomin" [paper] so paper specific questions can be asked there.

I will do this in the future, to release my code about the "unsupervised" and "semi-supervised" view learning part shown in the paper.