search

eth-sri / eran

ETH Robustness Analyzer for Deep Neural Networks
Apache License 2.0
313 stars 100 forks source link

Issue in the verification of a classifier with Softmax activation #42

Closed Tdai05 closed 3 years ago

Tdai05 commented 3 years ago

Dear Authors, Thanks very much for your great work with eran. I am trying to use it to verify a classifier. The model is similar as LeNet5 and there is a Softmax operation to map the prediction of each class to the range of [0,1]. I saw the tensorflow_translator.py include the operation of Softmax, However, using eran.analyze_box method and get a error which shows that Softmax operation can not be recognized. see below: AssertionError: the optimizer fordeeppoly doesn't know of the operation type Softmax

I have checked the model of cifar-10 which is shared by eran project and didn't find any Softmax operation in the ops. Have you applied Softmax for loss calculation and drop it for verification with eran? or something else? Would you please let me know your suggestion for this case? Thanks very much for the great help!

Best regards

GgnDpSngh commented 3 years ago

Hi there,

Thanks a lot for your interest in ERAN. Since the softmax layer is monotonic, it does not affect the verification results so it is a common practice to ignore it while running the verification algorithm. If you convert your model to onnx format then our onnx_translator will ignore softmax layer automatically.

Let me know if it works.

Cheers, Gagandeep Singh

Tdai05 commented 3 years ago

Got it and thanks very much for the help!

Cheers,