IBM / CLEVER-Robustness-Score

Codes for reproducing the robustness evaluation scores in “Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach,” ICLR 2018 ​​​​​​​
https://openreview.net/pdf?id=BkUHlMZ0b
Apache License 2.0
47 stars 18 forks source link

About default radius question #5

Open XiaoYigwr opened 5 years ago

XiaoYigwr commented 5 years ago

Thanks for your method, at the same time, I have a question to ask:

In your code

[_, loc, _] = weibull_min.fit(-np.array(grad_norm_set), c_init, optimizer=scipy_optimizer)#output:shape ,location ,scale

# Compute function value
values = classifier.predict(np.array([x]), logits=True)
value = values[:, pred_class] - values[:, target_class]

# Compute scores
s = np.min([-value[0] / loc, radius])

the parameter called "radius" are set 40, 2, 0.1 in the calculation of L1 norm, L2 norm and inf norm, I want to know whether there is a theoretical basis for these fixed defaults?

Wenrui Guo

pinyuchen commented 5 years ago

Hello Xiao,

These values you mentioned are the local radius parameters set for L1, L2 and Linfinity perturbation thresholds that are often used in the context of certified defenses in the literature. See an example of Linfinity threshold from https://arxiv.org/abs/1711.00851