About the paper

[fenss]

Hi, CorEx is a great work! And I am recently reading the paper , and I get trouble in understanding the optimization part in Sec. A. The original objective is like this: And because of: Then: Why the red-box part can be extracted? I think $p(x)p(y| x)$ is different from $p(x_i)p(y| x_i)$. Can you offer more details? Thanks!

[gregversteeg]

Yes. It might be easier to see it it like this:

That is, p(x_i,y) is just a marginal over the full p(x,y). We use this to pull out the expression in red.

I(Xi;Y)=\sum{x_i, y} p(x_i, y) \log (p(x_i, y) / p(xi) p(y)) = \sum{x, y} p(x, y) \log (p(x_i, y) / p(x_i) p(y))

[fenss]

I understood, thanks!