eth-sri / eran

ETH Robustness Analyzer for Deep Neural Networks
Apache License 2.0
323 stars 103 forks source link

Problems about refinezono and refinepoly #104

Open jiahaubai opened 2 years ago

jiahaubai commented 2 years ago

Hello,

Thank you for providing such a good tool for verification! This tool is user-friendly!

Here, I try to reproduce the result of the RefineZono and RefinePoly, but I have some problems with the command so, I would like to ask you some questions

[Q1] command of RefineZono

The paper introduces the parameters used in your experiment, such as theta, T, beta, and so on. However, I am still confused about the connection between the command usage in Github and the paper. For example, the usage in Github does not show the parameter, such as beta😅

In addition, I asked a similar question last year, so I also consider the command you provided in this issue python3 . --netname ../nets/mnist/mnist_relu_6_100.tf --dataset mnist --domain refinezono --epsilon 0.02 --num_test 100 --k 1 --refine_neurons --n_milp_refine 2 --timeout_milp 10 --timeout_lp 10

(But it does not seem thebeta or theta in the command too)

So, If I want to reproduce the result of refineZono in this figure.

image Could you please tell me all the commands you do in this experiment?

[Q2] find the paper of RefinePoly

I also see that you provide another verifying method - refinepoly

image

I try to find the paper in the SRI publications, but I think there is no corresponding match of the refinepoly in NIPS 2019 Is there a mistake in my search? Could you please tell me the paper link of RefinePoly?

I am sorry for giving you so much trouble. I am still looking forward to your reply!

Thanks! jiahau