search

danswer-ai / danswer

Gen-AI Chat for Teams - Think ChatGPT if it had access to your team's unique knowledge.
https://docs.danswer.dev/
Other
9.86k stars 1.12k forks source link

getting 507 Server Error: Insufficient Storage url when trying to ingest a pdf #839

Open sokoow opened 7 months ago

sokoow commented 7 months ago

Hi All,

When I try to add a file connector, I keep getting:

docker_compose-background-1     | requests.exceptions.HTTPError: 507 Server Error: Insufficient Storage for url: http://index:8081/document/v1/default/danswer_chunk/docid/fb20700b-9e98-5cc6-bbdb-c5bf5275525a

I'm using dev docker-compose file. Would you have an idea on what's this about? Perhaps it's trying to write something outside /opt/vespa/var ?

Weves commented 6 months ago

Hey @sokoow, Vespa doesn't like it when we approach the disk limit (it stops accepting new documents when total disk usage hits 75%). My first guess is that you might be running into that.

How much free storage space do you have on the machine you're running this on?

sokoow commented 6 months ago

hey @Weves , thanks for answering - that indeed might be the cause, I'm at 93% of my 500gigs which is still plenty of space, any ideas how to turn that condition on?

yuhongsun96 commented 6 months ago

If you're running in Docker, Vespa is only aware of the resources allocated to Docker, so if you use more than 75% of disk for Docker, it will start complaining. You could also up this value and rebuild but this causes some instability. If just a local test, you'll be fine though. If you deploy Danswer on a dedicated instance, you won't run into these issues anyhow.

We're generally more responsive on Slack or Discord if you want to join.

If you're setting up Danswer for your team, we (Chris and myself, the maintainers) are more than happy to help you set it up over a quick live call. If this is something you're interested in, you can book a time with us on my Cal!