search

zylon-ai / private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks
https://privategpt.dev
Apache License 2.0
54.11k stars 7.28k forks source link

Server fails if it receives 2 prompts simultaneously #1951

Open iulix21 opened 5 months ago

iulix21 commented 5 months ago

I have a problem, when i'm trying to post request twice from 2 devices server get fail and restarts my docker container. in the logs is writen about segmentation fault and not more information. request:

http://localhost:8003/v1/chat/completions
{
    "include_sources": true,
    "stream": false,
    "use_context": true,
    "messages": [
          ...
    ]
}

my pc parameters: CPU: i9-14gen GPT: nvidia RTX-4090 RAM: 64GB DDR5

jaluma commented 4 months ago

Could you give us more details: LLM backend, if it's running on Docker, etc? Please check https://github.com/zylon-ai/private-gpt/issues/1955 if you are using LlamaCPP

iulix21 commented 4 months ago

Could you give us more details: LLM backend, if it's running on Docker, etc? Please check #1955 if you are using LlamaCPP

I am using llama-cpp (but at the moment switched to ollama) with docker on win11 machine. model. mistral-7b-v0.2

jaluma commented 4 months ago

Could you share your stack trace? I just ran it using Docker+Ollama, and everything is working as expected (using macOS; I don't have access to a Windows machine).