TypeError: mistral isn't supported yet.

[stablegpt5279]

2023-10-17 07:46:51.279066: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT 2023-10-17 07:46:54,385 - INFO - run_localGPT.py:231 - Running on: cuda 2023-10-17 07:46:54,385 - INFO - run_localGPT.py:232 - Display Source Documents set to: False 2023-10-17 07:46:54,385 - INFO - run_localGPT.py:233 - Use history set to: False 2023-10-17 07:46:54,558 - INFO - SentenceTransformer.py:66 - Load pretrained SentenceTransformer: hkunlp/instructor-large load INSTRUCTOR_Transformer max_seq_length 512 2023-10-17 07:47:00,691 - INFO - run_localGPT.py:57 - Loading Model: TheBloke/Mistral-7B-Instruct-v0.1-GPTQ, on: cuda 2023-10-17 07:47:00,691 - INFO - run_localGPT.py:58 - This action can take a few minutes! 2023-10-17 07:47:00,691 - INFO - load_models.py:86 - Using AutoGPTQForCausalLM for quantized models Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. 2023-10-17 07:47:01,119 - INFO - load_models.py:93 - Tokenizer loaded Traceback (most recent call last): File "/content/localGPT/run_localGPT.py", line 268, in main() File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke return __callback(args, **kwargs) File "/content/localGPT/run_localGPT.py", line 239, in main qa = retrieval_qa_pipline(device_type, use_history, promptTemplate_type=model_type) File "/content/localGPT/run_localGPT.py", line 134, in retrieval_qa_pipline llm = load_model(device_type, model_id=MODEL_ID, model_basename=MODEL_BASENAME, LOGGING=logging) File "/content/localGPT/run_localGPT.py", line 67, in load_model model, tokenizer = load_quantized_model_qptq(model_id, model_basename, device_type, LOGGING) File "/content/localGPT/load_models.py", line 95, in load_quantized_model_qptq model = AutoGPTQForCausalLM.from_quantized( File "/usr/local/lib/python3.10/dist-packages/auto_gptq/modeling/auto.py", line 79, in from_quantized model_type = check_and_get_model_type(save_dir or model_name_or_path, trust_remote_code) File "/usr/local/lib/python3.10/dist-packages/auto_gptq/modeling/_utils.py", line 125, in check_and_get_model_type raise TypeError(f"{config.model_type} isn't supported yet.")

[stablegpt5279]

ImportError: Loading a GPTQ quantized model requires optimum (pip install optimum) and auto-gptq library (pip install auto-gptq)

[stablegpt5279]

ImportError: You need a version of auto_gptq >= 0.4.2 to use GPTQ: pip install --upgrade auto-gptq