A question about the method to get output from specific layer

[mountains-high]

Hello there, Thank you so much for the demo. It is quite helpful. In inverted_representation.py, I have a question about the method to get output from specific layer Line 50:

def get_output_from_specific_layer(self, x, selected_layer):
        layer_output = None
        for index, layer in enumerate(self.model.conv3.weight):
            #x = layer(x) #If I comment this line it works but I think commenting is wrong
            if str(index) == str(selected_layer):
                layer_output = x[:10]
                break
        return layer_output

If I don’t comment I get TypeError: 'Tensor' object is not callable. Am I missing something, I would be grateful for your suggestions. Thank you My model is :

Net(
  (conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))
  (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))
  (conv3): Conv2d(16, 120, kernel_size=(5, 5), stride=(1, 1))
  (fc1): Linear(in_features=120, out_features=84, bias=True)
  (fc2): Linear(in_features=84, out_features=10, bias=True)
)
State dict  conv1.weight     torch.Size([6, 1, 5, 5])
State dict  conv1.bias   torch.Size([6])
State dict  conv2.weight     torch.Size([16, 6, 5, 5])
State dict  conv2.bias   torch.Size([16])
State dict  conv3.weight     torch.Size([120, 16, 5, 5])
State dict  conv3.bias   torch.Size([120])
State dict  fc1.weight   torch.Size([84, 120])
State dict  fc1.bias     torch.Size([84])
State dict  fc2.weight   torch.Size([10, 84])
State dict  fc2.bias     torch.Size([10])

[utkuozbulak]

If you comment that line, the model doesn't do a forward pass until the selected_layer. Based on what you send, it is probably because your model does not have a separation of features/classifier for convolutional layers and fully connected layers so the forward pass fails. You have to edit the code to perform a proper forward pass.

[mountains-high]

Thank you for your suggestion. I changed the model. The mode looks like this:

Net(
  (features): Sequential(
    (0): Conv2d(1, 20, kernel_size=(5, 5), stride=(1, 1))
    (1): ReLU(inplace=True)
    (2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    (3): Conv2d(20, 50, kernel_size=(5, 5), stride=(1, 1))
    (4): ReLU(inplace=True)
    (5): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (classifier): Sequential(
    (0): Linear(in_features=800, out_features=500, bias=True)
    (1): ReLU(inplace=True)
    (2): Linear(in_features=500, out_features=10, bias=True)
  )
)
State dict  features.0.weight    torch.Size([20, 1, 5, 5])
State dict  features.0.bias      torch.Size([20])
State dict  features.3.weight    torch.Size([50, 20, 5, 5])
State dict  features.3.bias      torch.Size([50])
State dict  classifier.0.weight      torch.Size([500, 800])
State dict  classifier.0.bias    torch.Size([500])
State dict  classifier.2.weight      torch.Size([10, 500])
State dict  classifier.2.bias    torch.Size([10])

However, I get the same error

  x = layer(x)
TypeError: 'Tensor' object is not callable

The code which I used:

def get_output_from_specific_layer(self, x, layer_id):
        layer_output = None
        for index, layer in enumerate(self.model.features[3].weight):
            x = layer(x)
        #print(‘Layer is: ’, layer) #All layers will be printed out 
            if str(index) == str(layer_id):
                #x = layer(x)
                layer_output = x[0]
                #print('Layer is: ', layer)
                break
        return layer_output

I don’t know whether it’s correct or not, but it seems x = layer(x) not taking the weights of a specific layer. How do you think? Thank you for your time and considerations.

[utkuozbulak]

My guess: you are not iterating over weights correctly.

Change

        for index, layer in enumerate(self.model.features[3].weight):

to

        for index, layer in enumerate(self.model.features.weight):

[mountains-high]

Thank you. Now it is working! Have a nice day ~