Allow Debug Logging of Logits - Githubissues

dottxt-ai / outlines

Structured Text Generation

https://dottxt-ai.github.io/outlines/

Apache License 2.0

8.33k stars 425 forks source link

Allow Debug Logging of Logits #614

Open lapp0 opened 7 months ago

lapp0 commented 7 months ago

What behavior of the library made you think about the improvement?

If the model generates bad results we currently have no idea why. E.g.

How would you like it to behave?

For each generation step, we should have an optional debug log of

1) The softmaxed logits and decoded representation of the top N tokens
2) The softmaxed logits of all special tokens (namely EOS)

cpfiffer commented 1 day ago

Related https://github.com/lapp0/outlines/pull/35