microsoft / nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
https://nni.readthedocs.io
MIT License
14.06k stars 1.82k forks source link

after speed up the number of output dimention change #5717

Open m3bbass opened 11 months ago

m3bbass commented 11 months ago

Describe the issue: i used nni.compression.speedup and after the speed uo the number of output featuears change from 10 to 5 featurears only!!!!

GoogLeNet( (conv1): BasicConv2d( (conv): Conv2d(3, 32, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (maxpool1): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=True) (conv2): BasicConv2d( (conv): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (conv3): BasicConv2d( (conv): Conv2d(32, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (maxpool2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=True) (inception3a): Inception( (branch1): BasicConv2d( (conv): Conv2d(96, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (branch2): Sequential( (0): BasicConv2d( (conv): Conv2d(96, 48, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(48, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(48, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch3): Sequential( (0): BasicConv2d( (conv): Conv2d(96, 8, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(8, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(8, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(16, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch4): Sequential( (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True) (1): BasicConv2d( (conv): Conv2d(96, 16, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(16, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (inception3b): Inception( (branch1): BasicConv2d( (conv): Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (branch2): Sequential( (0): BasicConv2d( (conv): Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(64, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch3): Sequential( (0): BasicConv2d( (conv): Conv2d(128, 16, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(16, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(16, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(48, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch4): Sequential( (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True) (1): BasicConv2d( (conv): Conv2d(128, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (maxpool3): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=True) (inception4a): Inception( (branch1): BasicConv2d( (conv): Conv2d(240, 96, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (branch2): Sequential( (0): BasicConv2d( (conv): Conv2d(240, 48, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(48, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(48, 104, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(104, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch3): Sequential( (0): BasicConv2d( (conv): Conv2d(240, 8, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(8, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(8, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(24, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch4): Sequential( (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True) (1): BasicConv2d( (conv): Conv2d(240, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (inception4b): Inception( (branch1): BasicConv2d( (conv): Conv2d(256, 80, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(80, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (branch2): Sequential( (0): BasicConv2d( (conv): Conv2d(256, 56, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(56, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(56, 112, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(112, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch3): Sequential( (0): BasicConv2d( (conv): Conv2d(256, 12, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(12, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch4): Sequential( (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True) (1): BasicConv2d( (conv): Conv2d(256, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (inception4c): Inception( (branch1): BasicConv2d( (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (branch2): Sequential( (0): BasicConv2d( (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch3): Sequential( (0): BasicConv2d( (conv): Conv2d(256, 12, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(12, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch4): Sequential( (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True) (1): BasicConv2d( (conv): Conv2d(256, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (inception4d): Inception( (branch1): BasicConv2d( (conv): Conv2d(256, 56, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(56, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (branch2): Sequential( (0): BasicConv2d( (conv): Conv2d(256, 72, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(72, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(72, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(144, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch3): Sequential( (0): BasicConv2d( (conv): Conv2d(256, 16, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(16, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch4): Sequential( (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True) (1): BasicConv2d( (conv): Conv2d(256, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (inception4e): Inception( (branch1): BasicConv2d( (conv): Conv2d(264, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (branch2): Sequential( (0): BasicConv2d( (conv): Conv2d(264, 80, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(80, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(80, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch3): Sequential( (0): BasicConv2d( (conv): Conv2d(264, 16, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(16, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(16, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch4): Sequential( (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True) (1): BasicConv2d( (conv): Conv2d(264, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (maxpool4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=True) (inception5a): Inception( (branch1): BasicConv2d( (conv): Conv2d(416, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (branch2): Sequential( (0): BasicConv2d( (conv): Conv2d(416, 80, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(80, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(80, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(160, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch3): Sequential( (0): BasicConv2d( (conv): Conv2d(416, 16, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(16, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(16, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch4): Sequential( (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True) (1): BasicConv2d( (conv): Conv2d(416, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (inception5b): Inception( (branch1): BasicConv2d( (conv): Conv2d(416, 192, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (branch2): Sequential( (0): BasicConv2d( (conv): Conv2d(416, 96, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(96, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(96, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(192, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch3): Sequential( (0): BasicConv2d( (conv): Conv2d(416, 24, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(24, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) (1): BasicConv2d( (conv): Conv2d(24, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) (branch4): Sequential( (0): MaxPool2d(kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=True) (1): BasicConv2d( (conv): Conv2d(416, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (aux1): None (aux2): None (avgpool): AdaptiveAvgPool2d(output_size=(1, 1)) (dropout): Dropout(p=0.2, inplace=False) (fc): Linear(in_features=512, out_features=5, bias=True) )

Environment:

Configuration:

Log message:

How to reproduce it?: