Colab generates error

[kmkarakaya]

Colab generates error:

ValueError: Loading models/falcon-7b-instruct-GPTQ requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code=True to remove this error.

Details:

2023-08-14 09:42:41 INFO:Unwanted HTTP request redirected to localhost :) 2023-08-14 09:42:44 WARNING:The gradio "share link" feature uses a proprietary executable to create a reverse tunnel. Use it with care. 2023-08-14 09:42:46.457649: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so 2023-08-14 09:42:49 INFO:Loading falcon-7b-instruct-GPTQ... 2023-08-14 09:42:49 INFO:The AutoGPTQ params are: {'model_basename': 'gptq_model-4bit-64g', 'device': 'cuda:0', 'use_triton': False, 'inject_fused_attention': True, 'inject_fused_mlp': True, 'use_safetensors': True, 'trust_remote_code': False, 'max_memory': None, 'quantize_config': None, 'use_cuda_fp16': True} ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /content/text-generation-webui/server.py:1154 in │ │ │ │ 1151 │ │ update_model_parameters(model_settings, initial=True) # hija │ │ 1152 │ │ │ │ 1153 │ │ # Load the model │ │ ❱ 1154 │ │ shared.model, shared.tokenizer = load_model(shared.model_name │ │ 1155 │ │ if shared.args.lora: │ │ 1156 │ │ │ add_lora_to_model(shared.args.lora) │ │ 1157 │ │ │ │ /content/text-generation-webui/modules/models.py:74 in load_model │ │ │ │ 71 │ │ │ │ return None, None │ │ 72 │ │ │ 73 │ shared.args.loader = loader │ │ ❱ 74 │ output = load_func_maploader │ │ 75 │ if type(output) is tuple: │ │ 76 │ │ model, tokenizer = output │ │ 77 │ else: │ │ │ │ /content/text-generation-webui/modules/models.py:288 in AutoGPTQ_loader │ │ │ │ 285 def AutoGPTQ_loader(model_name): │ │ 286 │ import modules.AutoGPTQ_loader │ │ 287 │ │ │ ❱ 288 │ return modules.AutoGPTQ_loader.load_quantized(model_name) │ │ 289 │ │ 290 │ │ 291 def ExLlama_loader(model_name): │ │ │ │ /content/text-generation-webui/modules/AutoGPTQ_loader.py:56 in │ │ load_quantized │ │ │ │ 53 │ } │ │ 54 │ │ │ 55 │ logger.info(f"The AutoGPTQ params are: {params}") │ │ ❱ 56 │ model = AutoGPTQForCausalLM.from_quantized(path_to_model, params) │ │ 57 │ │ │ 58 │ # These lines fix the multimodal extension when used with AutoGPTQ │ │ 59 │ if hasattr(model, 'model'): │ │ │ │ /usr/local/lib/python3.10/dist-packages/auto_gptq/modeling/auto.py:79 in │ │ from_quantized │ │ │ │ 76 │ │ warmup_triton: bool = False, │ │ 77 │ │ kwargs │ │ 78 │ ) -> BaseGPTQForCausalLM: │ │ ❱ 79 │ │ model_type = check_and_get_model_type(save_dir or model_name_o │ │ 80 │ │ quant_func = GPTQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantiz │ │ 81 │ │ keywords = {key: kwargs[key] for key in signature(quant_func). │ │ 82 │ │ return quant_func( │ │ │ │ /usr/local/lib/python3.10/dist-packages/auto_gptq/modeling/_utils.py:123 in │ │ check_and_get_model_type │ │ │ │ 120 │ │ 121 │ │ 122 def check_and_get_model_type(model_dir, trust_remote_code=False): │ │ ❱ 123 │ config = AutoConfig.from_pretrained(model_dir, trust_remote_code=t │ │ 124 │ if config.model_type not in SUPPORTED_MODELS: │ │ 125 │ │ raise TypeError(f"{config.model_type} isn't supported yet.") │ │ 126 │ model_type = config.model_type │ │ │ │ /usr/local/lib/python3.10/dist-packages/transformers/models/auto/configurati │ │ on_auto.py:947 in from_pretrained │ │ │ │ 944 │ │ config_dict, unused_kwargs = PretrainedConfig.get_config_dict( │ │ 945 │ │ has_remote_code = "auto_map" in config_dict and "AutoConfig" i │ │ 946 │ │ has_local_code = "model_type" in config_dict and config_dict[" │ │ ❱ 947 │ │ trust_remote_code = resolve_trust_remote_code( │ │ 948 │ │ │ trust_remote_code, pretrained_model_name_or_path, has_loca │ │ 949 │ │ ) │ │ 950 │ │ │ │ /usr/local/lib/python3.10/dist-packages/transformers/dynamic_module_utils.py │ │ :553 in resolve_trust_remote_code │ │ │ │ 550 │ │ │ _raise_timeout_error(None, None) │ │ 551 │ │ │ 552 │ if has_remote_code and not has_local_code and not trust_remote_cod │ │ ❱ 553 │ │ raise ValueError( │ │ 554 │ │ │ f"Loading {model_name} requires you to execute the configu │ │ 555 │ │ │ " repo on your local machine. Make sure you have read the │ │ 556 │ │ │ " set the option trust_remote_code=True to remove this e │ ╰──────────────────────────────────────────────────────────────────────────────╯ ValueError: Loading models/falcon-7b-instruct-GPTQ requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code=True to remove this error.

[camenduru]

thanks for the info ❤ please test new v2.3