search

caikit / caikit-nlp

Apache License 2.0
12 stars 49 forks source link

duplicate PR for tokenization tasks request #331

Closed swith004 closed 9 months ago

swith004 commented 9 months ago

duplicate PR of 325 , which Added tokenization tasks in order to serve tokenization request and return the token count