Finetuning MAMBA on 25MB of data

johnma2006 / mamba-minimal

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

Apache License 2.0

2.54k stars 188 forks source link

Finetuning MAMBA on 25MB of data #30

Open omaruno opened 1 month ago

omaruno commented 1 month ago

How long it takes to finetune the model? Iam using 25MB of data (a super small dataset withrespect to the one real with around 4,5GB of data) and It take more than 1.30h to complete 1 epochs. Is it normal? Or maybe Iam doing something wrong?