search

Verified-Intelligence / auto_LiRPA

auto_LiRPA: An Automatic Linear Relaxation based Perturbation Analysis Library for Neural Networks and General Computational Graphs
https://arxiv.org/pdf/2002.12920
Other
290 stars 75 forks source link

Intermediate bounds #22

Closed nurlanov-zh closed 2 years ago

nurlanov-zh commented 2 years ago

Hello,

I am wondering if the library provides access to intermediate bounds too?

Thanks, Zhak

shizhouxing commented 2 years ago

Intermediate bounds for node can be accessed by node.lower and node.upper.

model._modules is a dictionary containing all the nodes. You may get a node by name: model._modules[name]. Or you may print items in model._modules to see what nodes you have.

nurlanov-zh commented 2 years ago

Thanks for the quick reply.

I have tried to access intermediate bounds by model._modules[name].lower or model._modules[name].upper, and it works for IBP and IBP+backward methods. However, for backward and CROWN-Optimized methods it gives the following error:

AttributeError: 'BoundRelu' object has no attribute 'lower'

AttributeError: 'BoundRelu' object has no attribute 'upper'

I assume that in CROWN the bounds for non-linearities have different structure. Is it still possible to access them for these methods?

huanzhang12 commented 2 years ago

@nurlanov-zh The lower and upper bound for Relu are not computed in this case for efficiency reasons. You can get the lower and upper bound for the layer before ReLU (typically a linear layer), and apply a ReLU function on these bounds.

nurlanov-zh commented 2 years ago

@huanzhang12 However, such obtained bounds will not be the same as in CROWN method, won't they? I wanted to look at intermediate bounds from particular methods. Is it possible to obtain those, even though the efficiency is reduced?