Maxpool Support

[MaxTorop]

Thank you for this wonderful library/paper!

I was wondering if you are planning to add support for maxpool2d in the near future, as I'm hoping to run SplineCAM for a resnet.

[AhmedImtiazPrio]

Hi Max! Unfortunately we might not be adding maxpool for exact computation in the near future :( because it prohibitively increases the computational complexity. However, we are adding an approximation library to Splinecam to allow using maxpools and approximating the number of regions. If you want to use Resnet out of the box then you would be able to use the approximation library to estimate the region density and related statistics without exactly computing the partitions. Would that be helpful? Were you thinking of doing exact visualizations for Resnet?

[MaxTorop]

Sorry, closing as not planned above was an accident.

Hi Ahmed, thanks for getting back to me. While exact visualization would have been a cherry on top, my main use case is actually counting the number of regions in a ball and computing statistics. So the approximation library sounds very helpful for my purposes ! Of course no rush, but is there a general date which you think this library would be added by ?

[AhmedImtiazPrio]

Thanks for your interest! I expect to have a version ready by the end of October.

[MaxTorop]

Hi Ahmed, I was wondering if you were still planning to release an approximation library which works with out of the box ResNets, with MaxPool (or if this is already online and I can't find it :-) ). I think the code in your grok-adversarial repo has some similar functionality, however I noticed the ResNet in that repo does not use MaxPool.

On a related note I see the ReadMe for the grok-adversarial RePo says "Coming Soon". Does this mean I should wait before using/building on the code in that RePo for my purposes --- or is the version uploaded online working as intended?

Thanks again for the excellent papers :-) I'm a big fan of of this line of work.

[AhmedImtiazPrio]

Hi Max,

Really appreciate your comments! The grok-adversarial repo is working, I'm yet to upload some MLP-MNIST codes that will complete the repo with all the experiments done in the paper, that's why it's still coming soon 😅

Right now that code will work for networks with maxpool but won't give appropriate numbers for the complexity induced by a maxpool layer I think. Was that something you needed? As in measure the non linearity due to maxpool? Do you wanna try it out and see if it matches your intuition? I'll try to give it some thought as well!

Best, Imtiaz

On Sat, Apr 13, 2024, 2:52 PM Max Torop @.***> wrote:

Hi Ahmed, I was wondering if you were still planning to release an approximation library which works with out of the box ResNets, with MaxPool (or if this is already online and I can't find it :-) ). I think the code in your grok-adversarial repo https://github.com/AhmedImtiazPrio/grok-adversarial/blob/main/local_complexity.py has some similar functionality, however I noticed the ResNet in that repo does not use MaxPool.

On a related note I see the ReadMe for the grok-adversarial RePo says "Coming Soon". Does this mean I should wait before using/building on the code in that RePo for my purposes --- or is the version uploaded online working as intended?

Thanks again for the excellent papers :-) I'm a big fan of of this line of work.

— Reply to this email directly, view it on GitHub https://github.com/AhmedImtiazPrio/splinecam/issues/8#issuecomment-2053743020, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH2F56NR5GILK3AISZ7MNNDY5GEGTAVCNFSM6AAAAAA33W3CACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJTG42DGMBSGA . You are receiving this because you commented.Message ID: @.***>

[MaxTorop]

Hi Ahmed,

Thank you for the fast response! Ok, that's great about the grok-adversarial repo, I will start playing around with it!

My interest isn't purely in the MaxPool itself per-se, but rather I was hoping to quantify the complexity in a "classic" ResNet architecture (i.e. conv --> maxpool --> a bunch of blocks --> avg pool --> classification head) --- only because that's what is usually trained on higher dim datasets in benchmarks. Since the MaxPool is near the beginning of the architecture, my intuition is that accounting/not accounting for it may have a significant effect on the post-hoc measured complexity, since latter layers may further sub-divide the polytopes induced at earlier layers.

In the meantime I can check some of my intuitions with your code on CIFAR10/100, since ResNets are generally not trained on these datasets with MaxPool anyway. I'll also check later on higher dimensional datasets with the "classic" MaxPool ResNet architecture and see if they match the intuitions I get on CIFAR10 --- and then later again if MaxPool complexity measurement functionality is developed.

Best,

Max