search

Mintplex-Labs / anything-llm

The all-in-one Desktop & Docker AI application with full RAG and AI Agent capabilities.
https://anythingllm.com
MIT License
19.22k stars 2.1k forks source link

[DOCS]: Add wording to Text splitting & Chunking Preferences to explain current strategy #1447

Open chrisn-au opened 3 months ago

chrisn-au commented 3 months ago

Description

Team, great project thx. Just getting up to speed. I have been struggling with understanding the appropriate strategy for chunking for my data. While I have no doubt there is a plan to add alternative chunking strategies in the short term I wanted to check you are ok with me doing a pull request to add some wording to the ext splitting & Chunking Preferences tab to explain that it uses LangChain's RecursiveCharacterTextSplit and a link to an explanation of that strategy - I am happy to do the pull request. Again awesome project - well done

timothycarambat commented 3 months ago

I think a small link in the UI would be appropriate, but moreso a new page on a new page in the https://docs.useanything.com/anythingllm-setup/embedder-configuration/overview page to explain more would be the best solution.

Those docs are open source so lengthy or wordy descriptions should for sure be offloaded there! https://github.com/Mintplex-Labs/anythingllm-docs