Mambular is a Python package that brings the power of Mamba architectures to tabular data, offering a suite of deep learning models for regression, classification, and distributional regression tasks. This includes models like Mambular, FT-Transformer, TabTransformer and tabular ResNets.
There is few optimizers which can show better generalization and training time: AdEMAMix and AdEMAMixShampoo
It can be included in repo or add some interface to change optimizer by yourself