network thinning allows bn only after conv

[gitti123]

I wanted to thin out a preact_resnet, but I got an assertionError from the function handle_bn_layers () (line 361 in thinning.py). The preact_resnet can be found here: https://github.com/kuangliu/pytorch-cifar/tree/master/models I think it is because of the expectation that the batchnormalization layer will come after the convolutional layer and not the other way round here. Is there a simple, fast solution to fix this?

[nzmora]

Hi @gitti123,

I think it is because of the expectation that the batchnormalization layer will come after the convolutional layer and not the other way round here

Exactly :-) The thinning code has hard-coded assumptions about the data-dependencies between (specific) module (operation) type, such as a BN module that follows a Convolution module. It searchs for sub-graphs such as Conv-BN and applies the thinning logic on these nodes (operations and tensors).
So the fix is to add logic to search of BN-Conv and apply the correct logic. This is an important extension of the thinning capabilities, but it's not currently a priority for us. If you want to try to do this yourself, I'd be happy to answer any questions you might have.

Thanks! Neta

[RizhaoCai]

Hi @gitti123,

I think it is because of the expectation that the batchnormalization layer will come after the convolutional layer and not the other way round here

Exactly :-) The thinning code has hard-coded assumptions about the data-dependencies between (specific) module (operation) type, such as a BN module that follows a Convolution module. It searchs for sub-graphs such as Conv-BN and applies the thinning logic on these nodes (operations and tensors). So the fix is to add logic to search of BN-Conv and apply the correct logic. This is an important extension of the thinning capabilities, but it's not currently a priority for us. If you want to try to do this yourself, I'd be happy to answer any questions you might have.

Thanks! Neta

If I am correct, there should be Conv-Bn combos in a network?

I got an AssertionError occurs

File "/home/**/***/distiller/distiller/thinning.py", line 357, in handle_bn_layers
    assert len(bn_layers) == 1
AssertionError

Is thie because I have no bn layers and hence "len(bn_layers) ==0"?