I ran the code successfully but can't receive the answer

[Jane1702]

[RobertoDonPedro]

I experience the same issue - No response, My setup:

• LInux Debian 11 (bullseye)
• 2 Nvidia T4 GPUs with 16GB GPU memory each
• cuBLAS enabled with CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python==0.1.83 --no-cache-dir

[faiconda]

I have the same problem

[PromtEngineer]

@Jane1702 @RobertoDonPedro @faiconda are you using the default model?

[RobertoDonPedro]

@Jane1702 @RobertoDonPedro @faiconda are you using the default model?

Yes, but I tried it with several other models as well, including the GPTQ models since I have 2 GPUs. But it just does not answer.

[Jane1702]

@Jane1702 @RobertoDonPedro @faiconda are you using the default model?

Yes , I used the default model

[RobertoDonPedro]

I just found the issue in my case. It was not the model, but this line in the run_localGPT.py file: query = input("\nEnter a query: ") It does not read the provided input and waits forever. If I replace this line manually in the code with e.g. query = "Summarize your content" (and add a breakat the end of the while loop) everything works as expected. I think this is caused by running localGPT with VSCode on a remote VM. However, still wondering what's the issue with VSCode in this regard.

[Jane1702]

Excuse-me , when I changed the code as you said , it finished the code but it didn't return the answer

[RobertoDonPedro]

Excuse-me , when I changed the code as you said , it finished the code but it didn't return the answer

..then the root of your issue is something else than in my case. You did ingest the data, right?

[Jane1702]

I have just run it successfully. I ran again with command prompt and it worked . Thank you very much for your help ^^ However , I don't know why it can't read some .txt files ?

[RobertoDonPedro]

Just solved my issue. It wasn't related to VS Code but to the fact that I called it from a nodejs app via shelljs and shelljs does not accept input. Now I call it natively via child_process.execFileSync with the stdio: 'inherit' parameter and now it works as expected. So in summary my issue was neither related to localGPT nor to VS Code but to they way I called localGPT. Potentially @Jane1702 you could also doublecheck the way you call it.

[Jane1702]

I don't know why but today I tried again and it worked normally . But it takes so long to have 1 answer (more than 20mins)

[KansaiTraining]

It took 20 minutes?!

(I am having the same problem)