Closed rsk2327 closed 1 week ago
The latest updates on your projects. Learn more about Vercel for Git ↗︎
@eyurtsev @baskaryan This metadata variable is a prerequisite for setting up the new implementation of the Sentence Window Retrieval method.
@hwchase17 @efriis Can i get a review on this?
Thank you for contributing to LangChain!
Description : Updated TextSplitter to include a new add_chunk_id argument to add a chunk_id variable into document metadata
This is a prerequisite for implementing the Sentence Window Retriever broadly across all databases as it helps easily identify neighboring chunks of text from the same text source
[x] Add tests and docs This is a simple argument addition thats False by default and has basic logic. Specific unit tests might not be necessary
[x] Lint and test: Run
make format
,make lint
andmake test
from the root of the package(s) you've modified.If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.