Open ba2512005 opened 7 months ago
Hi. I think I need to upgrade the automatic 1111's sd. I should separate its environment from lollms as they don't use the same python version anymore.
By the way. You can install it on your machine independantly and run it with the API server activated then lollms can use it directly. you can also use a remote version of SD, just set the url for the server in the artbot or lollms personality maker parameters.
Expected Behavior
The application should load properly.
Current Behavior
It seems when SD is enabled, it loads the SD model and fails to load the other normal LLM model and also fails to load the webui and it exits without error.
Steps to Reproduce
Possible Solution
I think it's failing because the GPU RAM is being overutilized on one gpu and not being split on the other GPU. Even with the device map to auto.
Context
running on a windows 11 system with ryzen 9 3950x, 128GB ram, rtx 3060 12GB and nvidia telsa p40
C:\Users\ba251\lollms-webui\scripts\windows>win_run.bat Starting LOLLMS Web UI... " " " /_\ /\ \ /_\ /_\ /_\ /\ \ " " /:/ / /::\ \ /:/ / /:/ / /::| | /::\ \ " " /:/ / /:/\:\ \ /:/ / /:/ / /:|:| | /:/\ \ \ " " /:/ / /:/ \:\ \ /:/ / /:/ / /:/|:|| \:\~\ \ \ " "/:// /:// \:_\ /:// /:// /:/ |::::_\ /\ \:\ \ _\ " "\:\ \ \:\ \ /:/ / \:\ \ \:\ \ \//~~/:/ / \:\ \:\ \// " " \:\ \ \:\ /:/ / \:\ \ \:\ \ /:/ / \:\ \:_\ " " \:\ \ \:\/:/ / \:\ \ \:\ \ /:/ / \:\/:/ / " " \:_\ \::/ / \:_\ \:_\ /:/ / \::/ / " " \// \// \// \// \// \// " By ParisNeo Trying to use Configuration at :global_paths_cfg.yaml global_paths_cfg.yaml found! Execution path : C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui Local zoos folder found ----------------------Paths information----------------------- personal_path:C:\Users\ba251\Documents\lollms personal_configuration_path:C:\Users\ba251\Documents\lollms\configs personal_databases_path:C:\Users\ba251\Documents\lollms\databases personal_models_path:C:\Users\ba251\Documents\lollms\models personal_user_infos_path:C:\Users\ba251\Documents\lollms\user_infos personal_trainers_path:C:\Users\ba251\Documents\lollms\trainers personal_trainers_path:C:\Users\ba251\Documents\lollms\trainers\gptqlora personal_data_path:C:\Users\ba251\Documents\lollms\data custom_personalities_path:C:\Users\ba251\Documents\lollms\custom_personalities custom_voices_path:C:\Users\ba251\Documents\lollms\custom_voices
----------------------Zoo information----------------------- bindings_zoo_path:C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo personalities_zoo_path:C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\personalities_zoo extensions_zoo_path:C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\extensions_zoo models_zoo_path:C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\models_zoo
Configuration file is very old. Replacing with default configuration {'version': 58, 'binding_name': None, 'model_name': None, 'headless_server_mode': False, 'allowed_origins': [], 'host': 'localhost', 'port': 9600, 'discussion_prompt_separator': '!@>', 'seed': -1, 'n_predict': 1024, 'ctx_size': 4084, 'min_n_predict': 512, 'temperature': 0.9, 'top_k': 50, 'top_p': 0.95, 'repeat_last_n': 40, 'repeat_penalty': 1.2, 'n_threads': 8, 'personalities': ['generic/lollms'], 'active_personality_id': 0, 'override_personality_model_parameters': False, 'extensions': [], 'user_name': 'user', 'user_description': '', 'use_user_name_in_discussions': False, 'user_avatar': 'default_user.svg', 'use_user_informations_in_discussion': False, 'db_path': 'database.db', 'debug': False, 'debug_log_file_path': '', 'auto_update': True, 'auto_sync_personalities': True, 'auto_sync_extensions': True, 'auto_sync_bindings': True, 'auto_sync_models': True, 'auto_save': True, 'auto_title': False, 'hardware_mode': 'nvidia-tensorcores', 'auto_show_browser': True, 'copy_to_clipboard_add_all_details': False, 'enable_voice_service': False, 'xtts_base_url': 'http://127.0.0.1:8020', 'auto_read': False, 'current_voice': None, 'current_language': 'en', 'enable_sd_service': False, 'sd_base_url': 'http://127.0.0.1:7860', 'enable_ollama_service': False, 'ollama_base_url': 'http://0.0.0.0:11434', 'enable_petals_service': False, 'petals_base_url': 'http://0.0.0.0:8010', 'enable_lollms_service': False, 'lollms_base_url': 'http://0.0.0.0:1234', 'elastic_search_service': False, 'elastic_search_url': 'http://0.0.0.0:9200', 'vllm_service': False, 'vllm_url': 'http://0.0.0.0:8000', 'vllm_model_path': 'mistralai/Mistral-7B-v0.1', 'media_on': False, 'audio_in_language': 'en-US', 'auto_speak': False, 'audio_out_voice': None, 'audio_pitch': 1, 'audio_auto_send_input': True, 'audio_silenceTimer': 5000, 'activate_ltm': False, 'summerize_discussion': False, 'max_summary_size': 512, 'data_vectorization_visualize_on_vectorization': False, 'use_files': True, 'data_vectorization_activate': True, 'data_vectorization_method': 'tfidf_vectorizer', 'data_visualization_method': 'PCA', 'data_vectorization_save_db': False, 'data_vectorization_chunk_size': 512, 'data_vectorization_overlap_size': 128, 'data_vectorization_nb_chunks': 2, 'data_vectorization_build_keys_words': False, 'data_vectorization_force_first_chunk': False, 'data_vectorization_make_persistance': False, 'activate_internet_search': False, 'internet_vectorization_chunk_size': 512, 'internet_vectorization_overlap_size': 128, 'internet_vectorization_nb_chunks': 2, 'internet_nb_search_pages': 3, 'internet_quick_search': False, 'internet_activate_search_decision': False, 'pdf_latex_path': None, 'positive_boost': None, 'negative_boost': None, 'force_output_language_to_be': None, 'fun_mode': False, 'show_code_of_conduct': True, 'activate_audio_infos': True, 'whisper_model': 'base'} Added entries : ['headless_server_mode', 'allowed_origins'], removed entries:[]
| | | | | | | | | | | | \ | || | | || || | | | | | | | | |__|||||||____|||__/ |___| Forked from Auto1111's Stable diffusion api Integration in lollms by ParisNeo using mix1009's sdwebuiapi Loading lollms_sd Running on windows venv "C:\Users\ba251\Documents\lollms\shared\auto_sd\venv\Scripts\Python.exe"
INCOMPATIBLE PYTHON VERSION
This program is tested with 3.10.6 Python, but you have 3.11.7. If you encounter an error with "RuntimeError: Couldn't install torch." message, or any other error regarding unsuccessful package (library) installation, please downgrade (or upgrade) to the latest version of 3.10 Python and delete current Python and "venv" folder in WebUI's directory.
You can download 3.10 Python from here: https://www.python.org/downloads/release/python-3106/
Alternatively, use a binary release of WebUI: https://github.com/AUTOMATIC1111/stable-diffusion-webui/releases
Use --skip-python-version-check to suppress this warning.
fatal: No names found, cannot describe anything. Python 3.11.7 | packaged by Anaconda, Inc. | (main, Dec 15 2023, 18:05:47) [MSC v.1916 64 bit (AMD64)] Version: 1.7.0 Commit hash: a94cbe85eea026af1f268e01c1ca3325868b1f99 Launching Web UI with arguments: --api no module 'xformers'. Processing without... no module 'xformers'. Processing without... No module 'xformers'. Proceeding without it. Style database not found: C:\Users\ba251\Documents\lollms\shared\auto_sd\styles.csv Loading weights [6ce0161689] from C:\Users\ba251\Documents\lollms\shared\auto_sd\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors Creating model from config: C:\Users\ba251\Documents\lollms\shared\auto_sd\configs\v1-inference.yaml Running on local URL: http://127.0.0.1:7860
To create a public link, set
share=True
inlaunch()
. Startup time: 14.5s (prepare environment: 3.1s, import torch: 4.1s, import gradio: 1.9s, setup paths: 1.4s, initialize shared: 0.3s, other imports: 1.1s, setup codeformer: 0.2s, load scripts: 1.3s, create ui: 0.5s, gradio launch: 0.4s, add APIs: 0.1s). Service is available. Lollms_core found in the app space. Pulling last lollms_core remote: Enumerating objects: 14, done. remote: Counting objects: 100% (14/14), done. remote: Compressing objects: 100% (5/5), done. remote: Total 9 (delta 6), reused 7 (delta 4), pack-reused 0 Unpacking objects: 100% (9/9), 840 bytes | 15.00 KiB/s, done. From https://github.com/ParisNeo/lollms 002102b..909ec70 main -> origin/main Updating 002102b..909ec70 Fast-forward configs/config.yaml | 5 ++++- lollms/configs/config.yaml | 5 ++++- lollms/paths.py | 1 + 3 files changed, 9 insertions(+), 2 deletions(-) safe_store_path found in the app space. Pulling last safe_store_path Already up to date. Bindings zoo found in your personal space. Pulling last bindings zoo Applying attention optimization: Doggettx... done. Already up to date. Personalities zoo found in your personal space. Pulling last personalities zoo Model loaded in 3.5s (load weights from disk: 0.7s, create model: 0.6s, apply weights to model: 1.9s, calculate empty prompt: 0.1s). error: The following untracked working tree files would be overwritten by merge: thinking_methodologies/mixture_of_experts_perspective_ai/assets/logo.png Please move or remove them before you merge. Aborting Updating 46dafae..e425cd4 Extensions zoo found in your personal space. Pulling last Extensions zoo Already up to date. Models zoo found in your personal space. Pulling last Models zoo Already up to date.Screenshots
If applicable, add screenshots to help explain the issue.