search

h2oai / h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://codellama.h2o.ai/
http://h2o.ai
Apache License 2.0
10.96k stars 1.2k forks source link

Error AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels #1383

Closed linhcentrio closed 5 months ago

linhcentrio commented 5 months ago

image image

(C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv) C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06>python generate.py --share=False --gradio_offline_level=1 --base_model=LuongNam/Vistral-7B-Chat-AWQ --score_model=None --load_awq=model --use_safetensors=True --prompt_type=mistral --score_model=None --load_awq=model --use_safetensors=True --prompt_type=llama2 --max_seq_len=4096 --load_4bit=True --dark=True --llamacpp_dict="{'n_gpu_layers':1}" --cli=False --gradio=True Fontconfig error: Cannot load default config file: No such file: (null) Using Model luongnam/vistral-7b-chat-awq fatal: not a git repository (or any of the parent directories): .git load INSTRUCTOR_Transformer max_seq_length 512 Starting get_model: LuongNam/Vistral-7B-Chat-AWQ Overriding max_seq_len -> 4096 Windows fatal exception: code 0xc0000139

Thread 0x00000b04 (most recent call first): File "", line 241 in _call_with_frames_removed File "", line 1176 in create_module File "", line 571 in module_from_spec File "", line 674 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\linear\exllama.py", line 6 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\linear__init.py", line 1 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "", line 241 in _call_with_frames_removed File "", line 992 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\base.py", line 14 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\mpt.py", line 1 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\init.py", line 1 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "", line 241 in _call_with_frames_removed File "", line 992 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\init__.py", line 2 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\loaders.py", line 100 in get_loaders File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2520 in get_model File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2370 in get_model_retry File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2038 in main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 691 in _CallAndUpdateTrace File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 475 in _Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 141 in Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\utils.py", line 65 in H2O_Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 12 in entrypoint_main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 16 in Windows fatal exception: code 0xc0000139

Thread 0x00000b04 (most recent call first): File "", line 241 in _call_with_frames_removed File "", line 1176 in create_module File "", line 571 in module_from_spec File "", line 674 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\linear\exllamav2.py", line 7 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\linear__init.py", line 2 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "", line 241 in _call_with_frames_removed File "", line 992 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\base.py", line 14 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\mpt.py", line 1 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\init.py", line 1 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "", line 241 in _call_with_frames_removed File "", line 992 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\init__.py", line 2 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\loaders.py", line 100 in get_loaders File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2520 in get_model File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2370 in get_model_retry File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2038 in main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 691 in _CallAndUpdateTrace File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 475 in _Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 141 in Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\utils.py", line 65 in H2O_Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 12 in entrypoint_main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 16 in Windows fatal exception: code 0xc0000139

Thread 0x00000b04 (most recent call first): File "", line 241 in _call_with_frames_removed File "", line 1176 in create_module File "", line 571 in module_from_spec File "", line 674 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\linear\gemm.py", line 8 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\linear__init.py", line 3 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "", line 241 in _call_with_frames_removed File "", line 992 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\base.py", line 14 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\mpt.py", line 1 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\init.py", line 1 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "", line 241 in _call_with_frames_removed File "", line 992 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\init__.py", line 2 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\loaders.py", line 100 in get_loaders File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2520 in get_model File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2370 in get_model_retry File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2038 in main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 691 in _CallAndUpdateTrace File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 475 in _Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 141 in Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\utils.py", line 65 in H2O_Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 12 in entrypoint_main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 16 in Windows fatal exception: code 0xc0000139

Thread 0x00000b04 (most recent call first): File "", line 241 in _call_with_frames_removed File "", line 1176 in create_module File "", line 571 in module_from_spec File "", line 674 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\linear\gemv.py", line 5 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\linear__init.py", line 4 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "", line 241 in _call_with_frames_removed File "", line 992 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\base.py", line 14 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\mpt.py", line 1 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\init.py", line 1 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "", line 241 in _call_with_frames_removed File "", line 992 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\init__.py", line 2 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\loaders.py", line 100 in get_loaders File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2520 in get_model File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2370 in get_model_retry File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2038 in main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 691 in _CallAndUpdateTrace File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 475 in _Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 141 in Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\utils.py", line 65 in H2O_Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 12 in entrypoint_main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 16 in Windows fatal exception: code 0xc0000139

Thread 0x00000b04 (most recent call first): File "", line 241 in _call_with_frames_removed File "", line 1176 in create_module File "", line 571 in module_from_spec File "", line 674 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\fused\norm.py", line 5 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\llama.py", line 11 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models__init__.py", line 2 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "", line 241 in _call_with_frames_removed File "", line 992 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq__init__.py", line 2 in File "", line 241 in _call_with_frames_removed File "", line 883 in exec_module File "", line 688 in _load_unlocked File "", line 1006 in _find_and_load_unlocked File "", line 1027 in _find_and_load File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\loaders.py", line 100 in get_loaders File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2520 in get_model File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2370 in get_model_retry File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 2038 in main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 691 in _CallAndUpdateTrace File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 475 in _Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\fire\core.py", line 141 in Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\utils.py", line 65 in H2O_Fire File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 12 in entrypoint_main File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\generate.py", line 16 in Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Overriding max_seq_len -> 4096 Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Overriding max_seq_len -> 4096 device_map: {'': 0} Fetching 8 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 8264.64it/s] Replacing layers...: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:10<00:00, 3.11it/s] Fusing layers...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:06<00:00, 5.28it/s] Model {'base_model': 'LuongNam/Vistral-7B-Chat-AWQ', 'base_model0': 'LuongNam/Vistral-7B-Chat-AWQ', 'tokenizer_base_model': '', 'lora_weights': '', 'inference_server': '', 'prompt_type': 'llama2', 'prompt_dict': {'promptA': '', 'promptB': '', 'PreInstruct': "[INST] <>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n<>\n\n", 'PreInput': None, 'PreResponse': '[/INST]', 'terminate_response': ['[INST]', ''], 'chat_sep': ' ', 'chat_turn_sep': ' ', 'humanstr': '[INST]', 'botstr': '[/INST]', 'generates_leading_space': False, 'system_prompt': "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.", 'can_handle_system_prompt': True}, 'visible_models': None, 'h2ogpt_key': None, 'load_8bit': False, 'load_4bit': True, 'low_bit_mode': 1, 'load_half': True, 'use_flash_attention_2': False, 'load_gptq': '', 'load_awq': 'model', 'load_exllama': False, 'use_safetensors': True, 'revision': None, 'use_gpu_id': True, 'gpu_id': 0, 'compile_model': None, 'use_cache': None, 'llamacpp_dict': {'n_gpu_layers': 1, 'n_gqa': 0, 'model_path_llama': '', 'model_name_gptj': '', 'model_name_gpt4all_llama': '', 'model_name_exllama_if_no_config': '', 'n_batch': 128}, 'rope_scaling': {}, 'max_seq_len': 4096, 'exllama_dict': {}, 'gptq_dict': {}, 'attention_sinks': False, 'sink_dict': {}, 'truncation_generation': False, 'hf_model_dict': {}} Begin auto-detect HF cache text generation models No loading model Systran/faster-whisper-large-v3 because is_encoder_decoder=True No loading model Systran/faster-whisper-medium because is_encoder_decoder=True No loading model stabilityai/stable-diffusion-2-inpainting because stabilityai/stable-diffusion-2-inpainting does not appear to have a file named config.json. Checkout 'https://huggingface.co/stabilityai/stable-diffusion-2-inpainting/main' for available files. No loading model lllyasviel/ControlNet because lllyasviel/ControlNet does not appear to have a file named config.json. Checkout 'https://huggingface.co/lllyasviel/ControlNet/main' for available files. No loading model diffusers/controlnet-depth-sdxl-1.0-small because Unrecognized model in diffusers/controlnet-depth-sdxl-1.0-small. Should have a model_type key in its config.json, or contain one of the following strings in its name: albert, align, altclip, audio-spectrogram-transformer, autoformer, bark, bart, beit, bert, bert-generation, big_bird, bigbird_pegasus, biogpt, bit, blenderbot, blenderbot-small, blip, blip-2, bloom, bridgetower, bros, camembert, canine, chinese_clip, clap, clip, clip_vision_model, clipseg, clvp, code_llama, codegen, conditional_detr, convbert, convnext, convnextv2, cpmant, ctrl, cvt, data2vec-audio, data2vec-text, data2vec-vision, deberta, deberta-v2, decision_transformer, deformable_detr, deit, deta, detr, dinat, dinov2, distilbert, donut-swin, dpr, dpt, efficientformer, efficientnet, electra, encodec, encoder-decoder, ernie, ernie_m, esm, falcon, fastspeech2_conformer, flaubert, flava, fnet, focalnet, fsmt, funnel, fuyu, git, glpn, gpt-sw3, gpt2, gpt_bigcode, gpt_neo, gpt_neox, gpt_neox_japanese, gptj, gptsan-japanese, graphormer, groupvit, hubert, ibert, idefics, imagegpt, informer, instructblip, jukebox, kosmos-2, layoutlm, layoutlmv2, layoutlmv3, led, levit, lilt, llama, llava, longformer, longt5, luke, lxmert, m2m_100, marian, markuplm, mask2former, maskformer, maskformer-swin, mbart, mctct, mega, megatron-bert, mgp-str, mistral, mixtral, mobilebert, mobilenet_v1, mobilenet_v2, mobilevit, mobilevitv2, mpnet, mpt, mra, mt5, musicgen, mvp, nat, nezha, nllb-moe, nougat, nystromformer, oneformer, open-llama, openai-gpt, opt, owlv2, owlvit, patchtsmixer, patchtst, pegasus, pegasus_x, perceiver, persimmon, phi, pix2struct, plbart, poolformer, pop2piano, prophetnet, pvt, qdqbert, qwen2, rag, realm, reformer, regnet, rembert, resnet, retribert, roberta, roberta-prelayernorm, roc_bert, roformer, rwkv, sam, seamless_m4t, seamless_m4t_v2, segformer, sew, sew-d, siglip, siglip_vision_model, speech-encoder-decoder, speech_to_text, speech_to_text_2, speecht5, splinter, squeezebert, swiftformer, swin, swin2sr, swinv2, switch_transformers, t5, table-transformer, tapas, time_series_transformer, timesformer, timm_backbone, trajectory_transformer, transfo-xl, trocr, tvlt, tvp, umt5, unispeech, unispeech-sat, univnet, upernet, van, videomae, vilt, vipllava, vision-encoder-decoder, vision-text-dual-encoder, visual_bert, vit, vit_hybrid, vit_mae, vit_msn, vitdet, vitmatte, vits, vivit, wav2vec2, wav2vec2-bert, wav2vec2-conformer, wavlm, whisper, xclip, xglm, xlm, xlm-prophetnet, xlm-roberta, xlm-roberta-xl, xlnet, xmod, yolos, yoso No loading model wangqixun/YamerMIX_v8 because wangqixun/YamerMIX_v8 does not appear to have a file named config.json. Checkout 'https://huggingface.co/wangqixun/YamerMIX_v8/main' for available files. No loading model lllyasviel/Annotators because lllyasviel/Annotators does not appear to have a file named config.json. Checkout 'https://huggingface.co/lllyasviel/Annotators/main' for available files. No loading model openai/whisper-base.en because is_encoder_decoder=True No loading model microsoft/speecht5_hifigan because The checkpoint you are trying to load has model type hifigan but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date. No loading model diffusers/controlnet-canny-sdxl-1.0 because Unrecognized model in diffusers/controlnet-canny-sdxl-1.0. Should have a model_type key in its config.json, or contain one of the following strings in its name: albert, align, altclip, audio-spectrogram-transformer, autoformer, bark, bart, beit, bert, bert-generation, big_bird, bigbird_pegasus, biogpt, bit, blenderbot, blenderbot-small, blip, blip-2, bloom, bridgetower, bros, camembert, canine, chinese_clip, clap, clip, clip_vision_model, clipseg, clvp, code_llama, codegen, conditional_detr, convbert, convnext, convnextv2, cpmant, ctrl, cvt, data2vec-audio, data2vec-text, data2vec-vision, deberta, deberta-v2, decision_transformer, deformable_detr, deit, deta, detr, dinat, dinov2, distilbert, donut-swin, dpr, dpt, efficientformer, efficientnet, electra, encodec, encoder-decoder, ernie, ernie_m, esm, falcon, fastspeech2_conformer, flaubert, flava, fnet, focalnet, fsmt, funnel, fuyu, git, glpn, gpt-sw3, gpt2, gpt_bigcode, gpt_neo, gpt_neox, gpt_neox_japanese, gptj, gptsan-japanese, graphormer, groupvit, hubert, ibert, idefics, imagegpt, informer, instructblip, jukebox, kosmos-2, layoutlm, layoutlmv2, layoutlmv3, led, levit, lilt, llama, llava, longformer, longt5, luke, lxmert, m2m_100, marian, markuplm, mask2former, maskformer, maskformer-swin, mbart, mctct, mega, megatron-bert, mgp-str, mistral, mixtral, mobilebert, mobilenet_v1, mobilenet_v2, mobilevit, mobilevitv2, mpnet, mpt, mra, mt5, musicgen, mvp, nat, nezha, nllb-moe, nougat, nystromformer, oneformer, open-llama, openai-gpt, opt, owlv2, owlvit, patchtsmixer, patchtst, pegasus, pegasus_x, perceiver, persimmon, phi, pix2struct, plbart, poolformer, pop2piano, prophetnet, pvt, qdqbert, qwen2, rag, realm, reformer, regnet, rembert, resnet, retribert, roberta, roberta-prelayernorm, roc_bert, roformer, rwkv, sam, seamless_m4t, seamless_m4t_v2, segformer, sew, sew-d, siglip, siglip_vision_model, speech-encoder-decoder, speech_to_text, speech_to_text_2, speecht5, splinter, squeezebert, swiftformer, swin, swin2sr, swinv2, switch_transformers, t5, table-transformer, tapas, time_series_transformer, timesformer, timm_backbone, trajectory_transformer, transfo-xl, trocr, tvlt, tvp, umt5, unispeech, unispeech-sat, univnet, upernet, van, videomae, vilt, vipllava, vision-encoder-decoder, vision-text-dual-encoder, visual_bert, vit, vit_hybrid, vit_mae, vit_msn, vitdet, vitmatte, vits, vivit, wav2vec2, wav2vec2-bert, wav2vec2-conformer, wavlm, whisper, xclip, xglm, xlm, xlm-prophetnet, xlm-roberta, xlm-roberta-xl, xlnet, xmod, yolos, yoso No loading model latent-consistency/lcm-lora-sdxl because latent-consistency/lcm-lora-sdxl does not appear to have a file named config.json. Checkout 'https://huggingface.co/latent-consistency/lcm-lora-sdxl/main' for available files. No loading model microsoft/speecht5_tts because is_encoder_decoder=True No loading model thibaud/controlnet-openpose-sdxl-1.0 because Unrecognized model in thibaud/controlnet-openpose-sdxl-1.0. Should have a model_type key in its config.json, or contain one of the following strings in its name: albert, align, altclip, audio-spectrogram-transformer, autoformer, bark, bart, beit, bert, bert-generation, big_bird, bigbird_pegasus, biogpt, bit, blenderbot, blenderbot-small, blip, blip-2, bloom, bridgetower, bros, camembert, canine, chinese_clip, clap, clip, clip_vision_model, clipseg, clvp, code_llama, codegen, conditional_detr, convbert, convnext, convnextv2, cpmant, ctrl, cvt, data2vec-audio, data2vec-text, data2vec-vision, deberta, deberta-v2, decision_transformer, deformable_detr, deit, deta, detr, dinat, dinov2, distilbert, donut-swin, dpr, dpt, efficientformer, efficientnet, electra, encodec, encoder-decoder, ernie, ernie_m, esm, falcon, fastspeech2_conformer, flaubert, flava, fnet, focalnet, fsmt, funnel, fuyu, git, glpn, gpt-sw3, gpt2, gpt_bigcode, gpt_neo, gpt_neox, gpt_neox_japanese, gptj, gptsan-japanese, graphormer, groupvit, hubert, ibert, idefics, imagegpt, informer, instructblip, jukebox, kosmos-2, layoutlm, layoutlmv2, layoutlmv3, led, levit, lilt, llama, llava, longformer, longt5, luke, lxmert, m2m_100, marian, markuplm, mask2former, maskformer, maskformer-swin, mbart, mctct, mega, megatron-bert, mgp-str, mistral, mixtral, mobilebert, mobilenet_v1, mobilenet_v2, mobilevit, mobilevitv2, mpnet, mpt, mra, mt5, musicgen, mvp, nat, nezha, nllb-moe, nougat, nystromformer, oneformer, open-llama, openai-gpt, opt, owlv2, owlvit, patchtsmixer, patchtst, pegasus, pegasus_x, perceiver, persimmon, phi, pix2struct, plbart, poolformer, pop2piano, prophetnet, pvt, qdqbert, qwen2, rag, realm, reformer, regnet, rembert, resnet, retribert, roberta, roberta-prelayernorm, roc_bert, roformer, rwkv, sam, seamless_m4t, seamless_m4t_v2, segformer, sew, sew-d, siglip, siglip_vision_model, speech-encoder-decoder, speech_to_text, speech_to_text_2, speecht5, splinter, squeezebert, swiftformer, swin, swin2sr, swinv2, switch_transformers, t5, table-transformer, tapas, time_series_transformer, timesformer, timm_backbone, trajectory_transformer, transfo-xl, trocr, tvlt, tvp, umt5, unispeech, unispeech-sat, univnet, upernet, van, videomae, vilt, vipllava, vision-encoder-decoder, vision-text-dual-encoder, visual_bert, vit, vit_hybrid, vit_mae, vit_msn, vitdet, vitmatte, vits, vivit, wav2vec2, wav2vec2-bert, wav2vec2-conformer, wavlm, whisper, xclip, xglm, xlm, xlm-prophetnet, xlm-roberta, xlm-roberta-xl, xlnet, xmod, yolos, yoso End auto-detect HF cache text generation models Begin auto-detect llama.cpp models End auto-detect llama.cpp models Running on local URL: http://0.0.0.0:7860

To create a public link, set share=True in launch(). Started Gradio Server and/or GUI: server_name: localhost port: None Use local URL: http://localhost:7860/ C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\pydantic_internal_fields.py:151: UserWarning: Field "modelname" has conflict with protected namespace "model".

You may be able to resolve this warning by setting model_config['protected_namespaces'] = (). warnings.warn( C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\pydantic_internal_fields.py:151: UserWarning: Field "modelnames" has conflict with protected namespace "model".

You may be able to resolve this warning by setting model_config['protected_namespaces'] = (). warnings.warn( OpenAI API URL: http://0.0.0.0:5000 INFO:name:OpenAI API URL: http://0.0.0.0:5000 OpenAI API key: EMPTY INFO:name:OpenAI API key: EMPTY INFO: 127.0.0.1:49988 - "GET / HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/index-fcfd0285.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/index-17c8506f.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/svelte/svelte.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-b80f4c40.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Index-2fc6d523.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /info HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /theme.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Button-c39ada54.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Blocks-cebf4cff.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Button-2a9911a9.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Blocks-4a034b77.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Index-b2efa79d.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-88ffb6d9.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index.svelte_svelte_type_style_lang-a5dff249.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/utils-013c0d40.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Example-0af5ce42.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-11e1685d.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Example.svelte_svelte_type_style_lang-ccc27f63.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-2abed479.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-815f98ca.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-a90cda25.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Example-55d94c71.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-f43ff834.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-99d07eaa.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-8bd4bdf3.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-7ab72019.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Textbox-4c6022f4.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Example-e8628e88.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/ModifyUpload-7d41bdb3.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/FileUpload-e08bb775.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Example-4f56d95a.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-6224b77c.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Image-7b829b60.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/ImageUploader-57e340a3.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Example-e441efe6.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-f5395e19.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Index-5538a1d8.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Tabs-e53cc473.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-dc08ef34.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Video-540eb7aa.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/index-909a842f.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Index-7ea7c4ca.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Example-aca1592e.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Index-6e9da292.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-8659c753.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /custom_component/cab0185bc1cc773fe94c716b52537c1a/component/style.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Example-3390cfd5.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/index-bea7352a.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Index-5676d3d3.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-61d7d897.css HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/prism-python-d46c2487.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Example-6bee1446.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Index-3b511d5c.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-7cbc3ec9.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-24a33ce1.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Index-dffef64a.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-49f5f400.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/BlockTitle-32c87174.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Info-bec0f8e8.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Example-b79eee1b.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-f48cb652.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/DropdownArrow-408065ba.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Example-84a3b179.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Index-44c9e154.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Index-0d7ad3de.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Textbox-e0b022b4.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Check-f7edb5d9.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49998 - "GET /assets/Index-6d139769.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Copy-a69620a8.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Example-e0df15d4.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-d16eb7b5.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/FileUpload-f8a75ea1.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/BlockLabel-d63363eb.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Empty-1a961007.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49998 - "GET /assets/File-e96cd7d6.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/ModifyUpload.svelte_svelte_type_style_lang-9dc8c218.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/IconButton-1cb16677.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/ModifyUpload-a854d542.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Upload-96564545.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/DownloadLink-eefeeb30.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49998 - "GET /assets/file-url-74140225.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Undo-6c110254.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/UploadText-755a169b.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Example-04e4c2ae.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Index-868193ec.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/ShareButton-3f30aaf7.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-b860828e.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49998 - "GET /assets/Image-6cbf81d1.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/ImageUploader-ba2fb6c5.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Image-74d97202.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/SelectSource-ad167e2c.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Example-b7360a9a.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49998 - "GET /assets/Index-8f0b4307.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Tabs-84ad15f9.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-d571a8b1.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Index-bed35146.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Index-366b1374.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/Trim-7bd312df.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49998 - "GET /assets/index-f931577a.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Example-63607f02.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-89ad857a.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/index-21d69e4c.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Video-ddf0a248.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/dsv-a37bb3db.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/Index-f2086bd3.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Example-7d6bb9b6.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49988 - "GET /assets/Index-ef88d4e9.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Example-c8c78d8d.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49996 - "GET /assets/index-b880a992.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/Index-6ed87e4b.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/Index-aa6434db.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /custom_component/cab0185bc1cc773fe94c716b52537c1a/component/index.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/logo-3707f936.svg HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /custom_component/cab0185bc1cc773fe94c716b52537c1a/component/Index-f4230f0b.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49997 - "GET /assets/module-a4efca6e.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49989 - "GET /assets/module-8e8d6730.js HTTP/1.1" 200 OK INFO: 127.0.0.1:49991 - "GET /assets/module-698c739c.js HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "POST /queue/join HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "GET /queue/data?session_hash=xx78wt93zfb HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "POST /queue/join HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "GET /queue/data?session_hash=xx78wt93zfb HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "POST /queue/join HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "GET /file%3DC%3A/Users/Admin/AppData/Local/Temp/gradio/ec85e4f8c437a42c40af22a043a2d44577a48853/human.jpg HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "GET /queue/data?session_hash=xx78wt93zfb HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "POST /queue/join HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "GET /queue/data?session_hash=xx78wt93zfb HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "POST /queue/join HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "GET /queue/data?session_hash=xx78wt93zfb HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "POST /queue/join HTTP/1.1" 200 OK INFO: 127.0.0.1:50004 - "GET /queue/data?session_hash=xx78wt93zfb HTTP/1.1" 200 OK thread exception: (<class 'AssertionError'>, AssertionError('AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels'), <traceback object at 0x0000027C8FB1C280>) make stop: (<class 'AssertionError'>, AssertionError('AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels'), <traceback object at 0x0000027C8FB1C280>) hit stop evaluate_nochat exception: AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels: ('', '', '', True, 'llama2', '{ \'PreInput\': None,\n \'PreInstruct\': \'[INST] <>\n\'\n \'You are a helpful, respectful and honest assistant. Always \'\n \'answer as helpfully as possible, while being safe. Your \'\n \'answers should not include any harmful, unethical, racist, \'\n \'sexist, toxic, dangerous, or illegal content. Please \'\n \'ensure that your responses are socially unbiased and \'\n \'positive in nature.\n\'\n \'\n\'\n \'If a question does not make any sense, or is not factually \'\n \'coherent, explain why instead of answering something not \'\n "correct. If you don\'t know the answer to a question, "\n "please don\'t share false information.\n"\n \'<>\n\'\n \'\n\',\n \'PreResponse\': \'[/INST]\',\n \'botstr\': \'[/INST]\',\n \'can_handle_system_prompt\': True,\n \'chat_sep\': \' \',\n \'chat_turn_sep\': \' \',\n \'generates_leading_space\': False,\n \'humanstr\': \'[INST]\',\n \'promptA\': \'\',\n \'promptB\': \'\',\n \'system_prompt\': \'You are a helpful, respectful and honest assistant. \'\n \'Always answer as helpfully as possible, while being \'\n \'safe. Your answers should not include any harmful, \'\n \'unethical, racist, sexist, toxic, dangerous, or illegal \'\n \'content. Please ensure that your responses are socially \'\n \'unbiased and positive in nature.\n\'\n \'\n\'\n \'If a question does not make any sense, or is not \'\n \'factually coherent, explain why instead of answering \'\n "something not correct. If you don\'t know the answer to a "\n "question, please don\'t share false information.",\n \'terminate_response\': [\'[INST]\', \'\']}', 0.1, 0.75, 40, 0, 1, 1024, 0, False, 600, 1.07, 1, False, True, '', '', 'LLM', True, 'Query', [], 10, True, 512, 'Relevant', ['All'], [], [], [], [], 'Pay attention and remember the information below, which will help to answer the question or imperative after the context ends.', 'According to only the information in the document sources provided within the context above, write an insightful and well-structured response to: ', 'In order to write a concise single-paragraph or bulleted list summary, pay attention to the following text.', 'Using only the information in the document sources above, write a condensed and concise summary of key results (preferably as bullet points).', 'Answer this question with vibrant details in order for some NLP embedding model to use that answer as better query than original question: ', 'auto', ['DocTR', 'Caption', 'ASR'], ['PyMuPDF'], ['Unstructured'], '.[]', 10, 'auto', [], '', False, '[]', '[]', 'reverse_ucurve_sort', 512, -1, -1, 'split_or_merge', '\n\n', 0, 'auto', False, False, 'None', 'None', [], 1, None, None, {'model': 'model', 'tokenizer': 'tokenizer', 'device': 'cuda', 'base_model': 'LuongNam/Vistral-7B-Chat-AWQ', 'tokenizer_base_model': '', 'lora_weights': '[None/Remove]', 'inference_server': '[None/Remove]', 'prompt_type': 'llama2', 'prompt_dict': {'promptA': '', 'promptB': '', 'PreInstruct': "[INST] <>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n<>\n\n", 'PreInput': None, 'PreResponse': '[/INST]', 'terminate_response': ['[INST]', ''], 'chat_sep': ' ', 'chat_turn_sep': ' ', 'humanstr': '[INST]', 'botstr': '[/INST]', 'generates_leading_space': False, 'system_prompt': "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.", 'can_handle_system_prompt': True}, 'visible_models': 0, 'h2ogpt_key': None}, {'MyData': [None, '001226ab-0263-4b78-8bfc-0b21af96f0f6', 'd1d1e0d2-578f-4617-9d7c-0b42af83482b']}, {'langchain_modes': ['UserData', 'MyData', 'LLM', 'Disabled'], 'langchain_mode_paths': {'UserData': None}, 'langchain_mode_types': {'UserData': 'shared', 'github h2oGPT': 'shared', 'DriverlessAI docs': 'shared', 'wiki': 'shared', 'wiki_full': '', 'MyData': 'personal', 'LLM': 'either', 'Disabled': 'either'}}, {'headers': '', 'host': 'localhost:7860', 'username': 'd1d1e0d2-578f-4617-9d7c-0b42af83482b', 'connection': 'keep-alive', 'content-length': '173', 'sec-ch-ua': '"Not A(Brand";v="99", "Google Chrome";v="121", "Chromium";v="121"', 'sec-ch-ua-platform': '"Windows"', 'sec-ch-ua-mobile': '?0', 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36', 'content-type': 'application/json', 'accept': '/', 'origin': 'http://localhost:7860', 'sec-fetch-site': 'same-origin', 'sec-fetch-mode': 'cors', 'sec-fetch-dest': 'empty', 'referer': 'http://localhost:7860/', 'accept-encoding': 'gzip, deflate, br', 'accept-language': 'en-US,en;q=0.9,vi;q=0.8', 'cookie': 'ajs_anonymous_id=32ab704f-489e-4a41-b1a2-7075e81b0999; _xsrf=2|bd58580b|357e1357e1bc5c5698f6550b7c9e44e6|1707032321; _hp2_id.1680123994=%7B%22userId%22%3A%225249589608497957%22%2C%22pageviewId%22%3A%226481230624182788%22%2C%22sessionId%22%3A%22374132275603419%22%2C%22identity%22%3Anull%2C%22trackerVersion%22%3A%224.0%22%7D; _hp2_ses_props.1680123994=%7B%22ts%22%3A1707356322916%2C%22d%22%3A%22localhost%22%2C%22h%22%3A%22%2F%22%7D', 'host2': '127.0.0.1'}, {}, [['hi', '']]) Traceback (most recent call last): File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\queueing.py", line 495, in call_prediction output = await route_utils.call_process_api( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\route_utils.py", line 230, in call_process_api output = await app.get_blocks().process_api( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\blocks.py", line 1590, in process_api result = await self.call_function( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\blocks.py", line 1188, in call_function prediction = await utils.async_iteration(iterator) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\utils.py", line 502, in async_iteration return await iterator.anext() File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\utils.py", line 495, in anext return await anyio.to_thread.run_sync( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run result = context.run(func, args) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\utils.py", line 478, in run_sync_iterator_async return next(iterator) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\utils.py", line 661, in gen_wrapper response = next(iterator) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gradio_runner.py", line 4444, in bot for res in get_response(fun1, history, chatbot_role1, speaker1, tts_language1, roles_state1, File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gradio_runner.py", line 4339, in get_response for output_fun in fun1(): File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 4535, in evaluate raise thread.exc File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\utils.py", line 472, in run self._return = self._target(self._args, self._kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 4697, in generate_with_exceptions func(*args, *kwargs) File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\base.py", line 86, in generate return self.model.generate(args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\transformers\generation\utils.py", line 1479, in generate return self.greedy_search( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\transformers\generation\utils.py", line 2340, in greedy_search outputs = self( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\transformers\models\mistral\modeling_mistral.py", line 1154, in forward outputs = self.model( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\fused\model.py", line 101, in forward h, _, past_key_value = layer( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\fused\block.py", line 64, in forward norm_out = self.norm_1(hidden_states) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\fused\norm.py", line 19, in forward assert AWQ_INSTALLED, ( AssertionError: AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels

linhcentrio commented 5 months ago

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. Setting pad_token_id to eos_token_id:32000 for open-end generation. thread exception: (<class 'AssertionError'>, AssertionError('AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels'), <traceback object at 0x000002658C3A6900>) make stop: (<class 'AssertionError'>, AssertionError('AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels'), <traceback object at 0x000002658C3A6900>) hit stop evaluate_nochat exception: AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels: ('', '', '', True, 'open_chat_correct', "{ 'PreInput': None,\n 'PreInstruct': 'GPT4 Correct User: ',\n 'PreResponse': 'GPT4 Correct Assistant:',\n 'botstr': 'GPT4 Correct Assistant:',\n 'can_handle_system_prompt': False,\n 'chat_sep': '<|end_of_turn|>',\n 'chat_turn_sep': '<|end_of_turn|>',\n 'generates_leading_space': False,\n 'humanstr': 'GPT4 Correct User: ',\n 'promptA': '',\n 'promptB': '',\n 'system_prompt': '',\n 'terminate_response': ['GPT4 Correct Assistant:', '<|end_of_turn|>']}", 0.1, 0.75, 40, 0, 1, 1024, 0, False, 600, 1.07, 1, False, True, '', '', 'LLM', True, 'Query', [], 10, True, 512, 'Relevant', ['All'], [], [], [], [], 'Pay attention and remember the information below, which will help to answer the question or imperative after the context ends.', 'According to only the information in the document sources provided within the context above, write an insightful and well-structured response to: ', 'In order to write a concise single-paragraph or bulleted list summary, pay attention to the following text.', 'Using only the information in the document sources above, write a condensed and concise summary of key results (preferably as bullet points).', 'Answer this question with vibrant details in order for some NLP embedding model to use that answer as better query than original question: ', 'auto', ['DocTR', 'Caption'], ['PyMuPDF'], ['Unstructured'], '.[]', 10, 'auto', [], '', False, '[]', '[]', 'reverse_ucurve_sort', 512, -1, -1, 'split_or_merge', '\n\n', 0, 'auto', False, False, 'None', None, [], 1, None, None, {'model': 'model', 'tokenizer': 'tokenizer', 'device': 'cuda', 'base_model': 'TheBloke/openchat-3.5-1210-AWQ', 'tokenizer_base_model': '', 'lora_weights': '[None/Remove]', 'inference_server': '[None/Remove]', 'prompt_type': 'open_chat_correct', 'prompt_dict': {'promptA': '', 'promptB': '', 'PreInstruct': 'GPT4 Correct User: ', 'PreInput': None, 'PreResponse': 'GPT4 Correct Assistant:', 'terminate_response': ['GPT4 Correct Assistant:', '<|end_of_turn|>'], 'chat_sep': '<|end_of_turn|>', 'chat_turn_sep': '<|end_of_turn|>', 'humanstr': 'GPT4 Correct User: ', 'botstr': 'GPT4 Correct Assistant:', 'generates_leading_space': False, 'system_prompt': '', 'can_handle_system_prompt': False}, 'visible_models': 0, 'h2ogpt_key': None}, {'MyData': [None, '1d1d536b-8747-49ac-80df-303e47a73b32', 'ef2cd6ca-2fd9-4718-a66f-cc69c383ab65']}, {'langchain_modes': ['UserData', 'MyData', 'LLM', 'Disabled'], 'langchain_mode_paths': {'UserData': None}, 'langchain_mode_types': {'UserData': 'shared', 'github h2oGPT': 'shared', 'DriverlessAI docs': 'shared', 'wiki': 'shared', 'wiki_full': '', 'MyData': 'personal', 'LLM': 'either', 'Disabled': 'either'}}, {'headers': '', 'host': 'localhost:7860', 'username': 'ef2cd6ca-2fd9-4718-a66f-cc69c383ab65', 'connection': 'keep-alive', 'content-length': '173', 'sec-ch-ua': '"Not A(Brand";v="99", "Google Chrome";v="121", "Chromium";v="121"', 'sec-ch-ua-platform': '"Windows"', 'sec-ch-ua-mobile': '?0', 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36', 'content-type': 'application/json', 'accept': '/', 'origin': 'http://localhost:7860', 'sec-fetch-site': 'same-origin', 'sec-fetch-mode': 'cors', 'sec-fetch-dest': 'empty', 'referer': 'http://localhost:7860/', 'accept-encoding': 'gzip, deflate, br', 'accept-language': 'en-US,en;q=0.9,vi;q=0.8', 'cookie': 'ajs_anonymous_id=32ab704f-489e-4a41-b1a2-7075e81b0999; _xsrf=2|bd58580b|357e1357e1bc5c5698f6550b7c9e44e6|1707032321; _hp2_ses_props.1680123994=%7B%22ts%22%3A1707356322916%2C%22d%22%3A%22localhost%22%2C%22h%22%3A%22%2F%22%7D; _hp2_id.1680123994=%7B%22userId%22%3A%225249589608497957%22%2C%22pageviewId%22%3A%221008626499202570%22%2C%22sessionId%22%3A%22374132275603419%22%2C%22identity%22%3Anull%2C%22trackerVersion%22%3A%224.0%22%7D', 'host2': '127.0.0.1'}, {}, [['hi', '']]) Traceback (most recent call last): File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\queueing.py", line 495, in call_prediction output = await route_utils.call_process_api( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\route_utils.py", line 230, in call_process_api output = await app.get_blocks().process_api( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\blocks.py", line 1590, in process_api result = await self.call_function( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\blocks.py", line 1188, in call_function prediction = await utils.async_iteration(iterator) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\utils.py", line 502, in async_iteration return await iterator.anext() File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\utils.py", line 495, in anext return await anyio.to_thread.run_sync( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\anyio_backends_asyncio.py", line 807, in run result = context.run(func, args) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\utils.py", line 478, in run_sync_iterator_async return next(iterator) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\gradio\utils.py", line 661, in gen_wrapper response = next(iterator) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gradio_runner.py", line 4444, in bot for res in get_response(fun1, history, chatbot_role1, speaker1, tts_language1, roles_state1, File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gradio_runner.py", line 4339, in get_response for output_fun in fun1(): File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 4535, in evaluate raise thread.exc File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\utils.py", line 472, in run self._return = self._target(self._args, self._kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\src\gen.py", line 4697, in generate_with_exceptions func(*args, *kwargs) File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\models\base.py", line 86, in generate return self.model.generate(args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\transformers\generation\utils.py", line 1479, in generate return self.greedy_search( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\transformers\generation\utils.py", line 2340, in greedy_search outputs = self( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\transformers\models\mistral\modeling_mistral.py", line 1154, in forward outputs = self.model( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, kwargs) File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\fused\model.py", line 101, in forward h, _, past_key_value = layer( File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\fused\block.py", line 64, in forward norm_out = self.norm_1(hidden_states) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) File "C:\Users\Admin\Downloads\h2ogpt-client-nightly-2024-02-06\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "c:\users\admin\downloads\h2ogpt-client-nightly-2024-02-06\autoawq\autoawq\awq\modules\fused\norm.py", line 19, in forward assert AWQ_INSTALLED, ( AssertionError: AWQ kernels could not be loaded. Please install them from https://github.com/casper-hansen/AutoAWQ_kernels

pseudotensor commented 5 months ago

Thanks, wasn't aware. Did you try:

pip install autoawq-kernels -c reqs_optional/reqs_constraints.txt

For windows see: https://github.com/casper-hansen/AutoAWQ_kernels/releases/

e.g. for cu121: https://github.com/casper-hansen/AutoAWQ_kernels/releases/download/v0.0.3/autoawq_kernels-0.0.3-cp310-cp310-win_amd64.whl

for cu118: https://github.com/casper-hansen/AutoAWQ_kernels/releases/download/v0.0.3/autoawq_kernels-0.0.3+cu118-cp310-cp310-win_amd64.whl