search

NON906 / sd-webui-chatgpt

This is a repository for conversations using OpenAI API (compatible with ChatGPT) or llama.cpp in Stable Diffusion web UI.
MIT License
42 stars 7 forks source link

where to put the modelfile #7

Open Cooly75 opened 11 months ago

Cooly75 commented 11 months ago

Hello

can someone tell me where to put the chatgpt files for offline usage.

thanks in advance.

NON906 commented 11 months ago

What does "the chatgpt files for offline usage" mean? Since it uses OpenAI API, it currently does not support offline. Does that files mean model files like distributed by HuggingFace?

Cooly75 commented 11 months ago

i do use stable diffusion on a offline computer, therefor any application who uses online ressources to work are in my case worthless, on the otherhand i do not like big tech who collects uncontrolled any data, unfortunate i do not use your tool, the idea is great, openai is also not very interessting it can not be used offline.the huggingface modell is a good idea how it can be integrated into your tool for offline usage? otherwise gpt4all or xgpt is a alternative, but it have to be trained by the user for a while.

NON906 commented 10 months ago

I tried GPT4All, but it's too slow.

In the first place, it requires more specifications than stable diffusion, so it is difficult to operate comfortably. If you try the following, you will see that it is difficult because just talking uses a large amount of VRAM. https://github.com/oobabooga/text-generation-webui

It seems better to simply use the OpenAI API.

NON906 commented 10 months ago

Sorry, I've changed my mind. I tried Llama.cpp and it worked to some extent. I have published a version under development in a branch called langchain, so please try it out.

I checked with the model file named "openchat_3.5.Q6_K.gguf" at the URL below. https://huggingface.co/TheBloke/openchat_3.5-GGUF/tree/main

NON906 commented 10 months ago

The corresponding functionality has also been reflected in the main branch. I would appreciate it if you could check it out.

Cooly75 commented 10 months ago

sorry to replay that late , i was not home, i gonna test it out and come back to you in the next days .

one other good modell is also find here: https://huggingface.co/LoneStriker/openchat_3.5-8.0bpw-h6-exl2 i already used it and works good. it have almost the stand as march 2023 of ChatGPT / Openai.

smal question where to put the downloaded modell openchat_3.5.Q6_K.gguf?

NON906 commented 10 months ago

smal question where to put the downloaded modell openchat_3.5.Q6_K.gguf?

As long as you specify the full path, you can place it anywhere.

Cooly75 commented 10 months ago

mmmm. i trty so far everythink but it do not work. no reaction,

Sans titre how you can see it is loading unending

but on the console nothing happens Sans titre1

What do i miss?

NON906 commented 10 months ago

It seems that the configuration file is not loading. What happens if you restart the webui? Also, what is the version of webui etc.?

Cooly75 commented 10 months ago

Its what i have done many times now , with no succes. version i use right now is up2date v1.7.0

NON906 commented 10 months ago

I'm sorry, but this is not happening here. What happens if you delete the files located in extensions\sd-webui-chatgpt\settings?

Cooly75 commented 10 months ago

is it possible that it have somethig to do with my system?

Graphic card PNY NVIDIA A16 / TCSA16M-PB 64gb & Graphic processor card Fujitsu NVIDIA Tesla A100 80GB

NON906 commented 10 months ago

It seems that the files cannot be read, so the GPU does not seem to be involved.

Cooly75 commented 10 months ago

i gonna reinstall the hole system this weekend , we will find what is the couse of it