search

IvanDrokin / torch-conv-kan

This project is dedicated to the implementation and research of Kolmogorov-Arnold convolutional networks. The repository includes implementations of 1D, 2D, and 3D convolutions with different kernels, ResNet-like and DenseNet-like models, training code based on accelerate/PyTorch, as well as scripts for experiments with CIFAR-10 and Tiny ImageNet.
MIT License
411 stars 31 forks source link

Kolmogorov-Arnold Transformer #12

Closed Adamdad closed 2 months ago

Adamdad commented 2 months ago

KAN was strong but faced scalability issues. We tackled this with 3 simple tricks. By combining KAN with Transformers, we've built a much stronger and more scalable model. 💪

📄 Paper: https://arxiv.org/abs/2409.10594 💻 Code: https://github.com/Adamdad/kat

IvanDrokin commented 2 months ago

@Adamdad Thanks for sharing, looks good! Congrats

Adamdad commented 2 months ago

Thank you for the nice words!!!!! Can have a try