search

YenRaven / annoy_ltm

annoy long term memory experiment for oobabooga/text-generation-webui
32 stars 3 forks source link

AttributeError: module 'modules.shared' has no attribute 'soft_prompt' #16

Closed bbecausereasonss closed 1 year ago

bbecausereasonss commented 1 year ago

Running on local URL: http://127.0.0.1:7861

To create a public link, set share=True in launch(). INFO:Loading askmyteapot_GPT4-x-AlpacaDente2-30b-4bit... INFO:The AutoGPTQ params are: {'model_basename': 'GPT4-x-AlpacaDente2-30b-act--true-0.1percdamp-ptb-4bit', 'device': 'cuda:0', 'use_triton': False, 'use_safetensors': True, 'trust_remote_code': False, 'max_memory': None, 'quantize_config': BaseQuantizeConfig(bits=4, group_size=-1, damp_percent=0.01, desc_act=False, sym=True, true_sequential=True, model_name_or_path=None, model_file_base_name=None)} WARNING:The safetensors archive passed at models\askmyteapot_GPT4-x-AlpacaDente2-30b-4bit\GPT4-x-AlpacaDente2-30b-act--true-0.1percdamp-ptb-4bit.safetensors does not contain metadata. Make sure to save your model with the save_pretrained method. Defaulting to 'pt' metadata. WARNING:skip module injection for FusedLlamaMLPForQuantizedModel not support integrate without triton yet. INFO:Loaded the model in 44.53 seconds.

failed to load character annoy metadata, generating from scratch... building annoy index took 0.018998384475708008 seconds... Traceback (most recent call last): File "C:\Users\xxxx\Deep\TextGen\installer_files\env\lib\site-packages\gradio\routes.py", line 427, in run_predict output = await app.get_blocks().process_api( File "C:\Users\xxxx\Deep\TextGen\installer_files\env\lib\site-packages\gradio\blocks.py", line 1323, in process_api result = await self.call_function( File "C:\Users\xxxx\Deep\TextGen\installer_files\env\lib\site-packages\gradio\blocks.py", line 1067, in call_function prediction = await utils.async_iteration(iterator) File "C:\Users\xxxx\Deep\TextGen\installer_files\env\lib\site-packages\gradio\utils.py", line 336, in async_iteration return await iterator.anext() File "C:\Users\xxxx\Deep\TextGen\installer_files\env\lib\site-packages\gradio\utils.py", line 329, in anext return await anyio.to_thread.run_sync( File "C:\Users\xxxx\Deep\TextGen\installer_files\env\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\Users\xxxx\Deep\TextGen\installer_files\env\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "C:\Users\xxxx\Deep\TextGen\installer_files\env\lib\site-packages\anyio_backends_asyncio.py", line 807, in run result = context.run(func, *args) File "C:\Users\xxxx\Deep\TextGen\installer_files\env\lib\site-packages\gradio\utils.py", line 312, in run_sync_iterator_async return next(iterator) File "C:\Users\xxxx\Deep\TextGen\text-generation-webui\modules\chat.py", line 332, in generate_chat_reply_wrapper for i, history in enumerate(generate_chat_reply(text, shared.history, state, regenerate, _continue, loading_message=True)): File "C:\Users\xxxx\Deep\TextGen\text-generation-webui\modules\chat.py", line 317, in generate_chat_reply for history in chatbot_wrapper(text, history, state, regenerate=regenerate, _continue=_continue, loading_message=loading_message): File "C:\Users\xxxx\Deep\TextGen\text-generation-webui\modules\chat.py", line 226, in chatbot_wrapper prompt = apply_extensions('custom_generate_chat_prompt', text, state, *kwargs) File "C:\Users\xxxx\Deep\TextGen\text-generation-webui\modules\extensions.py", line 193, in apply_extensions return EXTENSION_MAP[typ](args, kwargs) File "C:\Users\xxxx\Deep\TextGen\text-generation-webui\modules\extensions.py", line 80, in _apply_custom_generate_chat_prompt return extension.custom_generate_chat_prompt(text, state, kwargs) File "C:\Users\xxxx\Deep\TextGen\text-generation-webui\extensions\annoy_ltm\script.py", line 494, in custom_generate_chat_prompt return generator.custom_generate_chat_prompt(user_input, state, **kwargs) File "C:\Users\xxxx\Deep\TextGen\text-generation-webui\extensions\annoy_ltm\script.py", line 395, in custom_generate_chat_prompt if shared.soft_prompt: AttributeError: module 'modules.shared' has no attribute 'soft_prompt'

YenRaven commented 1 year ago

This should be fixed in main now. Thanks for the report!