jina-ai / late-chunking

Code for explaining and evaluating late chunking (chunked pooling)
Apache License 2.0
244 stars 29 forks source link

Add tests for chunking module #4

Closed violenil closed 1 month ago

violenil commented 1 month ago

This PR adds some more tests to the chunking methods, and adds back the semantic chunking logic.