Closed lixuyuan102 closed 3 months ago
Thank you for your interest in our work. In lines 73-81 of the code is the process of calculating the CLUB, which corresponds to the Technical details section on page 5 of the paper.
Thank you for your interest in our work. In lines 73-81 of the code is the process of calculating the CLUB, which corresponds to the Technical details section on page 5 of the paper.
The code 73-81 is used to compute the upper bound of MI for training the network of the main task. However, as the club paper 3.2 the neural club estimator requires separate training to approximate p(y|x) which allows the estimator to give a correct upper bound of MI. I'm just confused as to whether the estimator without being individually trained can give an upper bound of MI that supports the decoupling task.
Mutual_info_reg is actually a trainable module (e.g., the theta parameter of Eqs. 4 and 5 of the paper), although the inputs are features of the main network, and Table IV of the paper demonstrates the effectiveness of this module.
Mutual_info_reg is actually a trainable module (e.g., the theta parameter of Eqs. 4 and 5 of the paper), although the inputs are features of the main network, and Table IV of the paper demonstrates the effectiveness of this module.
Get it! Thanks for your reply.
Thanks for sharing your code. However, the club estimator seems not to be trained. It seems like the 'kl_dis' function in Mutual_info_reg is useless.