c0sogi / llama-api

An OpenAI-like LLaMA inference API
MIT License
111 stars 9 forks source link

model_definitions.py #12

Closed Dougie777 closed 1 year ago

Dougie777 commented 1 year ago

I know its probably easy for everyone else but I am struggling every time I add a new model to test. I often get "model not found" when it seems it is there. It would be a huge help if it didn't just return model not found, but the exact path and filename it is trying to load. Either in the terminal window or return info.

ps. when will the new branch be ready that handles these definitions better?

Thanks so much, Doug

c0sogi commented 1 year ago

When you start the application, you will see the parameter definitions of the different models highlighted in blue text against a white background. This includes the model_path.

I am currently considering adding an argument to customize the model_path location (--model-dir ./models). The default is './models', which represents a relative path. In fact, when navigating to the model file, it explores several possible locations. It tries to find a path that matches model_path from project sub and cwd based relative paths (it explores models, models/ggml, and models/gptq!).

To answer your question, yes. I'm going to add some additional code to tell the console where it tried to search, and this will be added to master branch soon.

c0sogi commented 1 year ago

d2a5c11e9abbe4b3abf2da0329bd05d94a934fd1

Dougie777 commented 1 year ago

Thank you so much !!