search

hotg-ai / proc-blocks

Common processing blocks used with your Runes.
https://hotg-ai.github.io/proc-blocks
9 stars 3 forks source link

can handle empty sentences #10

Closed Mohit0928 closed 3 years ago

Mohit0928 commented 3 years ago

Now Tokenizer can safely handle empty sentences

Michael-F-Bryan commented 3 years ago

Note that this left the assert!(!input_text_1.is_empty(), "Sentence 1 is empty") assertions in the code, so we can't actually support empty strings. Do you know why we need them in the first place?