search

codelion / optillm

Optimizing inference proxy for LLMs
Apache License 2.0
1.6k stars 128 forks source link

Fix confidence calculation #46

Closed jovanwongzixi closed 1 month ago

jovanwongzixi commented 1 month ago

probs.size() should take in dimension -1 instead of 1