search

benbrandt / text-splitter

Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from Rust and Python.
MIT License
235 stars 15 forks source link

Prep CodeSplitter Python bindings and release #206

Closed benbrandt closed 2 months ago

benbrandt commented 2 months ago

Marks the code splitter ready and also sets up the Python bindings so it can be called from there.

It also supports passing tree-sitter grammars from Python, since they are both using the same C bindings under the hood anyway.

codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.65%. Comparing base (690812a) to head (301c60d). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #206 +/- ## ======================================= Coverage 99.65% 99.65% ======================================= Files 11 11 Lines 2036 2036 ======================================= Hits 2029 2029 Misses 7 7 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.