search

philschmid / llm-sagemaker-sample

Apache License 2.0
37 stars 17 forks source link

training zephyr-7b #10

Closed piyushgit011 closed 6 months ago

piyushgit011 commented 6 months ago

hey, @philschmid i have trained zephyr-7b on qlora but when i am trying to deploy the llm then i am getting a problem :- image so, i couldn't create the endpoint. can anyone explain why it is happening. how can i solve this issue?

piyushgit011 commented 6 months ago

it's saying unsupported model type mistral.

piyushgit011 commented 6 months ago

this is where the errors starts:- image

piyushgit011 commented 6 months ago

now, i am getting this error , after changing gpu. image

philschmid commented 6 months ago

Which version of TGI are you using?

piyushgit011 commented 6 months ago

tgi 0.8.2 which should i use for this @philschmid ? trained on transformers_version = '4.28'

philschmid commented 6 months ago

can you please try 1.1.0 or 1.3.3

piyushgit011 commented 6 months ago

@philschmid can this be the problem as in config.json model architecture and model type is mistral? image