search

Atinoda / text-generation-webui-docker

Docker variants of oobabooga's text-generation-webui, including pre-built images.
GNU Affero General Public License v3.0
389 stars 74 forks source link

Is Mistral supported on default version? #26

Closed jaredquekjz closed 11 months ago

jaredquekjz commented 11 months ago

Hi I'm trying to use one of the Mistral derived model (which works on the non-docker text-gen) but I keep getting this error message. Things seem to work with the llama model? Am I doing something wrong or is Mistral not supported on the default version? I tried with the Nightly version but it seems to fail as well.

text-generation-webui  | To create a public link, set `share=True` in `launch()`.
text-generation-webui  | 2023-10-12 13:35:19 INFO:Loading Mistral-7B-OpenOrca...
text-generation-webui  | 2023-10-12 13:35:19 ERROR:Failed to load the model.
text-generation-webui  | Traceback (most recent call last):
text-generation-webui  |   File "/app/modules/ui_model_menu.py", line 198, in load_model_wrapper
text-generation-webui  |     shared.model, shared.tokenizer = load_model(shared.model_name, loader)
text-generation-webui  |   File "/app/modules/models.py", line 78, in load_model
text-generation-webui  |     output = load_func_map[loader](model_name)
text-generation-webui  |   File "/app/modules/models.py", line 122, in huggingface_loader
text-generation-webui  |     config = AutoConfig.from_pretrained(path_to_model, trust_remote_code=params['trust_remote_code'])
text-generation-webui  |   File "/venv/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1039, in from_pretrained
text-generation-webui  |     config_class = CONFIG_MAPPING[config_dict["model_type"]]
text-generation-webui  |   File "/venv/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 734, in __getitem__
text-generation-webui  |     raise KeyError(key)
text-generation-webui  | KeyError: 'mistral'
text-generation-webui  |
Atinoda commented 11 months ago

I have updated the docker image to v1.7 and successfully tested the default variant with mistralai/Mistral-7B-Instruct-v0.1. Please pull the latest version and give it a go! Exciting to see a really open source model released.

I used these settings in my test: image