search

VuBacktracking / mamba-text-classification

Text Classification using Mamba Model
MIT License
13 stars 2 forks source link

Mamba is not longer work with current code #2

Open TinChung41 opened 1 month ago

TinChung41 commented 1 month ago

hi Vu, I'm working on Mamba text-classification right now and a few month back your code and this code work normaly

https://colab.research.google.com/drive/13EC5kbiZmtmFqBOsTW7j-A8JEVGEhvWg?usp=sharing

But as I running it couple week ago, it give out multiples error that doesn't make sense when in the requirement it use old version of mamba-ssm and transformer. I been on this rabbit hole for a while since there no current official document for mama-ssm yet.

VuBacktracking commented 1 month ago

I updated the MambaConfig, you can check it now. I think it works well now. But if you still have a problem, can you tell me the details of yours?

TinChung41 commented 3 weeks ago

Hi, thank you for responsding, last time I try to use newer releases of mamba since it been 6 months but end up getting error so I open this issue

I try to install the new updated repo of your but still running into Import error. Specifically, If I use mamba-ssm==1.2.0.post1 from the requirements.txt it give this error (even after use pip install mamba-ssm==1.2.0.post1 --no-cache-dir)

and when I install new releases of mamba-ssm I get: 

Token is valid (permission: write).
Your token has been saved to /home/tinchung/.cache/huggingface/token
Login successful
Traceback (most recent call last):
  File "/home/tinchung/Documents/GitHub/mamba-text-classification/trainer.py", line 21, in <module>
    model = MambaTextClassification.from_pretrained("state-spaces/mamba-130m")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tinchung/Documents/GitHub/mamba-text-classification/mamba/model.py", line 62, in from_pretrained
    config = MambaConfig(**config_data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: MambaConfig() takes no arguments

So I kind of need to know which version you are using since old I can get fix the import error on the old mamba realease

sanjayagra commented 1 week ago

hi Vu, Can you please fix the above error (shared by TinChung41), I am also facing the same error?

Token is valid (permission: write). Your token has been saved to /home/tinchung/.cache/huggingface/token Login successful Traceback (most recent call last): File "/home/tinchung/Documents/GitHub/mamba-text-classification/trainer.py", line 21, in model = MambaTextClassification.from_pretrained("state-spaces/mamba-130m") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/tinchung/Documents/GitHub/mamba-text-classification/mamba/model.py", line 62, in from_pretrained config = MambaConfig(**config_data) ^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: MambaConfig() takes no arguments