predict function now accept this inputs:
texts (str, List[str], List[List[str]]) — The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings (pretokenized string).If the sequences are provided as list of strings (pretokenized), you must set is_split_into_words=True (to lift the ambiguity with a batch of sequences)
is_split_into_words - indicate if the texts input is tokenized
if nlp is not None:
tokenized_texts = tokenize_with_spacy(batch['text'], nlp)
else:
tokenized_texts = batch
tokenized_texts['offset_mapping'] = [(list(zip(range(len(tokens)), range(1, 1 + len(tokens)))))
for tokens in tokenized_texts['tokens']]
predict function now accept this inputs:
texts (str, List[str], List[List[str]])
— The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings (pretokenized string).If the sequences are provided as list of strings (pretokenized), you must set is_split_into_words=True (to lift the ambiguity with a batch of sequences)is_split_into_words
- indicate if the texts input is tokenizedUsage:
Note: no changes to
CorefResult
. solved with: