search

ScrapeGraphAI / Scrapegraph-ai

Python scraper based on AI
https://scrapegraphai.com
MIT License
14.41k stars 1.17k forks source link

Support for OpenAI Assistants API #629

Open matthewgertner opened 1 week ago

matthewgertner commented 1 week ago

The Assistants API has support for uploading files into a server-side vector store (https://platform.openai.com/docs/assistants/tools/file-search). This would eliminate the need for chunking files while scraping.

On the other hand, I don't know if other systems support server-side RAG so this might be OpenAI specific functionality.

VinciGit00 commented 1 week ago

Do you have the lang chain implementation?

matthewgertner commented 1 week ago

The PR is here https://github.com/langchain-ai/langchain/pull/21484