search

ParisNeo / lollms-webui

Lord of Large Language Models Web User Interface
https://lollms.com
Apache License 2.0
4.27k stars 537 forks source link

Saving binding settings for huggingface results in 404 error message #472

Open ba2512005 opened 8 months ago

ba2512005 commented 8 months ago

Expected Behavior

Saving the bindings should result in a 200 message on the server

Current Behavior

INFO: 127.0.0.1:35924 - "GET /update_binding_settings HTTP/1.1" 404 Not Found

Steps to Reproduce

click the little circle icon next to the personalities model in the discussion tab, change the binding settings and click save.

Possible Solution

Looks like the bindings API path has changed according to the fastAPI documentation http://localhost:9600/docs#/default/serve_bindings_bindings__path__get

Update the path for saving the bindings.

Context

Running on Ubuntu 22.04

ParisNeo commented 8 months ago

Can you confirm that this is fixed?

ba2512005 commented 8 months ago

I’ll have to confirm later on, my computer is apart right now, working to upgrade it.

I should have it done in the next day or so iA

On Mon, Jan 15, 2024 at 5:32 PM Saifeddine ALOUI @.***> wrote:

Can you confirm that this is fixed?

— Reply to this email directly, view it on GitHub https://github.com/ParisNeo/lollms-webui/issues/472#issuecomment-1892832969, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEF52KNWH3534QSNZNHVO73YOWU6ZAVCNFSM6AAAAABB2DAW6SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJSHAZTEOJWHE . You are receiving this because you authored the thread.Message ID: @.***>

ba2512005 commented 8 months ago

@ParisNeo The bindings setting saving look good now. However I am getting a No model selected error when trying to initiate a discussion. please see the trace below.

I think it requires low_cpu_mem to be set to true for applying the device_map. Why is this a requirement?

Recovering generation config /home//ai/personal/models/awq/Nous-Hermes-2-Mixtral-8x7B-SFT-AWQ Creating model /home//ai/personal/models/awq/Nous-Hermes-2-Mixtral-8x7B-SFT-AWQ Using device map: balanced Passing along a device_map requires low_cpu_mem_usage=True

Configuration model_name set to Nous-Hermes-2-Mixtral-8x7B-SFT-AWQ Configuration model_name updated INFO: 127.0.0.1:38134 - "POST /update_setting HTTP/1.1" 200 OK INFO: 127.0.0.1:34066 - "GET / HTTP/1.1" 304 Not Modified INFO: connection closed Client bBnmQNXNW8V_TWhCAAAB disconnected INFO: 127.0.0.1:34066 - "GET /assets/index-d3156257.js HTTP/1.1" 304 Not Modified INFO: 127.0.0.1:34066 - "GET /assets/index-a2ed5f05.css HTTP/1.1" 304 Not Modified INFO: 127.0.0.1:34066 - "GET /socket.io/?EIO=4&transport=polling&t=OqYHrvP HTTP/1.1" 200 OK INFO: 127.0.0.1:34066 - "GET /get_config HTTP/1.1" 200 OK Client 9M900wmwdHaYM6SsAAAD connected INFO: 127.0.0.1:34066 - "POST /socket.io/?EIO=4&transport=polling&t=OqYHrvr&sid=f-81bH09aGDOzmYQAAAC HTTP/1.1" 200 OK INFO: ('127.0.0.1', 34082) - "WebSocket /socket.io/?EIO=4&transport=websocket&sid=f-81bH09aGDOzmYQAAAC" [accepted] INFO: connection open INFO: 127.0.0.1:34084 - "GET /socket.io/?EIO=4&transport=polling&t=OqYHrvs&sid=f-81bH09aGDOzmYQAAAC HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /list_databases HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /socket.io/?EIO=4&transport=polling&t=OqYHrwC&sid=f-81bH09aGDOzmYQAAAC HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /get_lollms_webui_version HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /list_bindings HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /disk_usage HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /ram_usage HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /vram_usage HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /list_extensions HTTP/1.1" 200 OK Listing all personalities OK INFO: 127.0.0.1:34084 - "GET /get_all_personalities HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /get_available_models HTTP/1.1" 200 OK Listing models ok INFO: 127.0.0.1:34084 - "GET /list_models HTTP/1.1" 200 OK Getting active model ok INFO: 127.0.0.1:34084 - "GET /get_active_model HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /get_model_status HTTP/1.1" 200 OK INFO: 127.0.0.1:34084 - "GET /list_discussions HTTP/1.1" 200 OK Loading discussion for client 9M900wmwdHaYM6SsAAAD ... ok INFO: 127.0.0.1:34084 - "GET /get_current_personality_files_list HTTP/1.1" 200 OK

ba2512005 commented 8 months ago

When I do set it to try it with low_cpu_mem set to true, the model loads but i get the following error:

RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.