shivani-athavale
commented
2 months ago Hi @yanhuixie,
I tried to reproduce this on my end, but I do not get this error. I am able to import modeling_mamba with transformers version 4.39.
Perhaps you can try to follow the instructions at https://github.com/amd/RyzenAI-SW/tree/main/example/transformers/models/llm again, maybe some step was missed?
The modeling_mamba.py file is located at this path https://github.com/amd/RyzenAI-SW/tree/main/example/transformers/models/llm/mamba. After following the installation steps, you can check that this path to the file should be added to the PYTHONPATH in your environment.
All steps are based on these docs. https://ryzenai.docs.amd.com/en/latest/inst.html https://ryzenai.docs.amd.com/en/latest/llm_flow.html https://github.com/amd/RyzenAI-SW/blob/main/example/transformers/models/llm/docs/README.md The previous steps did not encounter any errors until the python run_stmoothquantit.py -- model_namellama-2-7b chat -- task quantize encountered this error. After a simple investigation, I found that mamba_ssm is a tool specifically designed for nVidia's GPU and CUDA. Why is it being used here? Any suggestion is appreciated.
(ryzenai-transformers) PS D:\path\to\RyzenAI\transformers\models\llm> python run_smoothquant.py --model_name llama-2-7b-chat --task quantize Traceback (most recent call last): File "D:\path\to\RyzenAI\transformers\models\llm\run_smoothquant.py", line 12, in
import llm_eval
File "D:\path\to\RyzenAI\transformers\tools\llm_eval.py", line 41, in
from modeling_mamba import MambaForCausalLM
ModuleNotFoundError: No module named 'modeling_mamba'