search

zbh2047 / L_inf-dist-net-v2

[ICLR 2022] Training L_inf-dist-net with faster acceleration and better training strategies
22 stars 4 forks source link

Up and Down Variables #3

Open gwmdunda opened 10 months ago

gwmdunda commented 10 months ago

Dear Authors,

Thank you for your works. May I ask why we need up and down variables in the model?

zbh2047 commented 10 months ago

Hi, thanks for your interest in our work. Here we add up and down variables in order to support the interval bound propagation in computing the certified radius for L-infinity distance layers. You may not use it if the certified radius is computed simply based on the Lipschitz property. You will need up and down variables for hybrid architectures, e.g., a MLP is built on top of an L-infinity distance net. Hope this can address your question.

gwmdunda commented 10 months ago

Thank you for your answer. May I also ask why there is a negative sign in the forward() function (x, lower, upper = -x, -upper, -lower)

zbh2047 commented 10 months ago

Hi, could you please provide the location of this line of code?

gwmdunda commented 10 months ago

It is in ell_inf_models.py line 43

zbh2047 commented 10 months ago

The model implemented in this repo follows the original ICML paper, where in Section 3.2 g(x) = (−x_1^{(L)}, −x_2^{(L)}, ..., −x_M^{(L)}). So this line of code takes negative for x, lower, upper variables.

gwmdunda commented 9 months ago

I am a bit confused since in ICML paper you need to take the negative sign, but in your ICLR paper, the negative sign is not needed.