Mintplex-Labs / anything-llm

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
https://anythingllm.com
MIT License
26.54k stars 2.65k forks source link

[FEAT]: Website scraping depth #1190

Closed shatfield4 closed 5 months ago

shatfield4 commented 6 months ago

What would you like to see?

We should create a data connector that allows scraping websites with a configurable depth

flefevre commented 6 months ago

Is there any documentation on website querying? I have seen anythingllm is compatible with Web site but no way to find the documentation. Thanks for the links/help