search

badripatro / simba

Simba
156 stars 15 forks source link

EinFFT as a channel mixer for other architectures #14

Open Xynonners opened 1 month ago

Xynonners commented 1 month ago

Hi there,

I read the research paper, however one thing I was wondering is what would happen if one were to mate EinFFT to a different architecture, for example standard transformers instead of Mamba in place of the MLP/FF? I would understand if the EinFFT were built for Mamba specifically but would be interested to find the answer to this question.

Thanks