search

yujonglee / eval

Evaluate your LLM apps, RAG pipeline, any generated text, and more!
MIT License
0 stars 0 forks source link

Better text formatting on thought generation #99

Open yujonglee opened 1 year ago

yujonglee commented 1 year ago

Current approach:

https://github.com/fastrepl/fastrepl/blob/f18c86dddef268f111bf4328c77e820eaa0499e1/fastrepl/eval/model/llm_head_cot.py#L27 https://github.com/fastrepl/fastrepl/blob/f18c86dddef268f111bf4328c77e820eaa0499e1/fastrepl/eval/model/llm_head_cot.py#L54

Maybe this approach is more reliable(ask model to wrap text):

https://platform.openai.com/docs/guides/gpt-best-practices/strategy-give-gpts-time-to-think https://docs.anthropic.com/claude/docs/give-claude-room-to-think-before-responding