I propose adding a new parameter $overlap = 0 to the splitDocument() method in DocumentSplitter and making the function consider the specified number of characters in overlapping the previous chunk in to the current chunk.
This would allow for the chunks being created to overlap a specified amount of characters to optimize for RAG use cases
I propose adding a new parameter
$overlap = 0
to thesplitDocument()
method inDocumentSplitter
and making the function consider the specified number of characters in overlapping the previous chunk in to the current chunk.This would allow for the chunks being created to overlap a specified amount of characters to optimize for RAG use cases