search

run-llama / sec-insights

A real world full-stack application using LlamaIndex
https://www.secinsights.ai/
MIT License
2.32k stars 631 forks source link

add try/catch for file exists error #69

Closed sourabhdesai closed 10 months ago

sourabhdesai commented 10 months ago

Fix for issue mentioned here.

I think what's occurring is that since we're spinning up multiple threads here, we're calling split_by_sentence_tokenizer within the lifespan method multiple times which are getting executed simultaneously. This seems to lead to a race condition where if one finishes before the other method calls, the other method calls raise a FileExistsError complaining that the file already exists. This shouldn't be a big problem given that we've actually downloaded the file that is being used internally by NLTK.

vercel[bot] commented 10 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
llama-app-frontend ✅ Ready (Inspect) Visit Preview Nov 3, 2023 5:39pm