search

AntonioTepsich / Convolutional-KANs

This project extends the idea of the innovative architecture of Kolmogorov-Arnold Networks (KAN) to the Convolutional Layers, changing the classic linear transformation of the convolution to learnable non linear activations in each pixel.
MIT License
779 stars 76 forks source link

Implementation of loss function #18

Open whaot opened 3 weeks ago

whaot commented 3 weeks ago

In addition to cross entropy loss, there are also 2 regularization terms for KAN model. Regularization loss is defined for class KAN_Convolutional_Layer in kan_convolutional/KANConv.py. def regularization_loss(self, regularize_activation=1.0, regularize_entropy=1.0): return sum( layer.regularization_loss(regularize_activation, regularize_entropy) for layer in self.layers) However, in experiment_28x28.ipynb, only cross-entropy loss seems to work. criterion_KKAN_Convolutional_Network = nn.CrossEntropyLoss() Is it because I missed something?

AlexBodner commented 2 weeks ago

Excuse us, the regularization is implemented in another branch (the one called regularization loss), but now it has been outdated because of recent changes that we've done. With some minor tweaks it should work.