search

yujonglee / eval

Evaluate your LLM apps, RAG pipeline, any generated text, and more!
MIT License
0 stars 0 forks source link

How is the tokenizer used? #92

Closed krrishdholakia closed 1 year ago

krrishdholakia commented 1 year ago

https://github.com/fastrepl/fastrepl/blob/4c80ccd008341aa2081d8edb543645c171e4a6d7/fastrepl/llm.py#L168

yujonglee commented 1 year ago

This is for generating logit_bias, which helps reliability of simple classification.

https://github.com/fastrepl/fastrepl/blob/7ef185e12b41d29aca869ea4440e5068a3344365/fastrepl/eval/model/utils.py#L17-L33

krrishdholakia commented 1 year ago

thanks!

yujonglee commented 1 year ago

See https://github.com/BerriAI/litellm/issues/312 for further work on LiteLLM.