!mistral-demo $7B_DIR issue

mistralai / mistral-inference

Official inference library for Mistral models

https://mistral.ai/

Apache License 2.0

9.16k stars 803 forks source link

!mistral-demo $7B_DIR issue #154

Open shaimaa0000 opened 1 month ago

shaimaa0000 commented 1 month ago

Hi , When I try to run this command '!mistral-demo $7B_DIR', I encounter an issue related to the GPU. Could you please suggest a solution? I am using Google Colab

issue pic

sachinsshetty commented 1 month ago

replace 7B_DIR with M7B_DIR at all places looks like an issue with setting environment variables with numeric character. faced the same issue on ubuntu 22.04. replacing the variable solved the problem.

patrickvonplaten commented 1 month ago

Also note that not every GPU will work with the required version of xformers. If you're having a T4 on google colab, maybe try to manually install a lower version of xformers

kartikWorld commented 2 weeks ago

you need to have a NVIDIA A100 machine to fine tune. I faced this earlier with V100. After switching to A100, it worked.