AntonioTepsich / Convolutional-KANs

This project extends the idea of the innovative architecture of Kolmogorov-Arnold Networks (KAN) to the Convolutional Layers, changing the classic linear transformation of the convolution to learnable non linear activations in each pixel.
MIT License
779 stars 76 forks source link

How to set the number of output channels for a tensor? #4

Closed SkyWxxk closed 6 months ago

SkyWxxk commented 6 months ago

Firstly, thanks to authors' KAN-CNN works. But I was confused that the output channel number of the tensor was directly multiplied by the n_convs parameter, rather than the expected output channel number. Here is the demo:

import torch
from torch import nn
import torch.nn.functional as F
from kan_convolutional.KANConv import KAN_Convolutional_Layer

class KANC_MLP(nn.Module):
    def __init__(self,device: str = 'cpu'):
        super().__init__()
        self.conv1 = KAN_Convolutional_Layer(
            n_convs = 3,
            kernel_size= (3,3),
            padding =(1,1),
            device = device
        )
        self.conv2 = KAN_Convolutional_Layer(
            n_convs = 5,
            kernel_size = (3,3),
            padding =(1,1),
            device = device
        )
        self.pool1 = nn.MaxPool2d(
            kernel_size=(2, 2)
        )
        self.flat = nn.Flatten() 
        self.linear1 = nn.Linear(3675, 256)
        self.linear2 = nn.Linear(256, 10)

    def forward(self, x):
        x = self.conv1(x)
        print(f'After conv1: {x.size()}')  # Print size after conv1
        x = self.pool1(x)
        print(f'After pool1: {x.size()}')  # Print size after pool1
        x = self.conv2(x)
        print(f'After conv2: {x.size()}')  # Print size after conv2
        x = self.pool1(x)
        print(f'After pool2: {x.size()}')  # Print size after pool2
        return x

def main():
    model = KANC_MLP(device="cpu")
    x = torch.randn(1, 3, 32, 32)
    output = model(x)
if __name__ == "__main__":
    main()

The result shows here:

After conv1: torch.Size([1, 9, 32, 32]) After pool1: torch.Size([1, 9, 16, 16]) After conv2: torch.Size([1, 45, 16, 16]) After pool2: torch.Size([1, 45, 8, 8])

We can see that the channel was just multiplied by "n_convs". How can we get the desired output channel? Thanks!

AlexBodner commented 6 months ago

Hello, the responsible for it here. In our implementation, you especify the ammount of convolutions that you want in a layer, so it applies each conv to each entering channel, resulting in a channels * convs output. Hope that this clarifies it. Regards, Alex