ksanman / AllMiniLML6v2Sharp

net standard 2.1 Library for running Sentence Transformers All-MiniLM-L6-v2 from C#.
MIT License
12 stars 6 forks source link

Handle text longer than 512 tokens. #3

Closed ksanman closed 5 months ago

ksanman commented 6 months ago

Long text chunks seem to make the model input throw exceptions. This does not happen with the python lib, same model. Investigate the cause and add either a. Truncation to long text or friendly error handling.