search

openvinotoolkit / openvino_tokenizers

OpenVINO Tokenizers extension
Apache License 2.0
20 stars 16 forks source link

Fix infinite loop in split for empty split_pattern #233

Closed pavel-esir closed 2 weeks ago

pavel-esir commented 2 weeks ago

When split pattern is empty both RE2 and PCRE2 get_next_match return matching curr_begin and curr_end and enter infinite loop since start was not incremented. Need to exit cycle when that happens.

CVS-150821