KOR extraction doesn't yields reproducible results

eyurtsev / kor

LLM(😽)

https://eyurtsev.github.io/kor/

MIT License

1.61k stars 88 forks source link

KOR extraction doesn't yields reproducible results #280

Closed nans-here closed 5 months ago

nans-here commented 6 months ago

I am doing KOR extraction with following specifications -

Using Object from kor.nodes to specify schema
Specifying ID and description of the object and attributes
Using gpt 3.5 with seed=42, temperature=0, top_p=0.1

I am getting different results in each iteration.

eyurtsev commented 6 months ago

Could you share a snippet of what you're doing? No non determinism on the kor side that I'm aware of. I'd also print out the prompt sent to the LLM (or you can trace with langsmith to check what's going on)

eyurtsev commented 5 months ago

Closing this is this is not likely to be a Kor issue.