Closed peterzpy closed 3 years ago
We intially employed the L1Out formulation to estimate the mutual information, as in On Variational Bounds of Mutual Information. But as shown in our pretrained model, with suitable hyperparameters, the model without mutual information can achieve comparable performance. So mutual information may not be necessary for our model.
Well, thanks for your reply.
Hi! As shown in table1, the mutual information module improves the performance well. But I can't find the implementation of this module in your code. Since the general mutual information estimators can only get the lower bound of the real mutual information, I have no idea how do you minimize the mutual information between different parts. Do you use the methods like CLUB to model the upper bound of the mutual information? Looking forward to your reply, thx!