Open chrisn-au opened 3 months ago
I think a small link in the UI would be appropriate, but moreso a new page on a new page in the https://docs.useanything.com/anythingllm-setup/embedder-configuration/overview page to explain more would be the best solution.
Those docs are open source so lengthy or wordy descriptions should for sure be offloaded there! https://github.com/Mintplex-Labs/anythingllm-docs
Description
Team, great project thx. Just getting up to speed. I have been struggling with understanding the appropriate strategy for chunking for my data. While I have no doubt there is a plan to add alternative chunking strategies in the short term I wanted to check you are ok with me doing a pull request to add some wording to the ext splitting & Chunking Preferences tab to explain that it uses LangChain's RecursiveCharacterTextSplit and a link to an explanation of that strategy - I am happy to do the pull request. Again awesome project - well done