How to use this?

[CyberTimon]

Hello! This project looks great! But I don't know how to use it. When I launch main.py I only see this:

cybertimon@server:~/Repositorys/BrainChulo$ python main.py INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 0 tokens INFO:llama_index.token_counter.token_counter:> [query] Total LLM token usage: 19546 tokens INFO:llama_index.token_counter.token_counter:> [query] Total embedding token usage: 0 tokens Please do nothing else to this text. --------------------------------------------

Launching webui.py also does nothing. It says for more info I should check the documentation. But where is it? Thanks

[iGavroche]

What Generative Text service are you using? Oobabooga's?

[CyberTimon]

Yes the latest Oobabooga (With the api) Many people including me opened issues in oobabooga that the api is buggy / broken, so could this be the issue? What should happen?

[iGavroche]

This is how I start mine: python server.py --model TheBloke_vicuna-7B-1.1-GPTQ-4bit-128g --wbits 4 --groupsize 128 --model_type llama --listen --no-stream --verbose --xformers

Look at the --listen and in our case, we would want --no-stream as well to ensure we get responses in one call. Also, make sure to create you .env file with the correct hostname and port.

Let me know if that makes sense, and if it does, feel free to close the issue once you've resolved your problem.

[CyberTimon]

Sadly this doesn't fix the issue. By the way, what is supposed to happen next?

[iGavroche]

In one Terminal window you'd run your WebUI server, and in the other one you'd run main.py. The objective is for main.py to load a document as context to your Generative Chat server, then it would ask a question about something specific to that context and obtain a proper answer.

[iGavroche]

@CyberTimon I've updated the README.md file to provide more clarity based on your feedback: https://github.com/CryptoRUSHGav/BrainChulo/commit/ff53e37deea94277fae5aef5357bb5a394e9ece9

I am closing this issue now.