This is super neat, restricting logits to valid tokens is a neat approach. Writing an entirely new json streaming parser to support it even more so :D
I'm currently working on https://github.com/iantbutler01/asimov which is what is being called a semi autonomous agent framework these days. For obvious reasons structured output is super important.
I was looking at approaches to help with that and I really like this, but without a permissive license I'm wary of including it in my project because by default no license means all rights reserved here on Github, even though it seems like your intent is for people to make use of this!
Also I'd be happy to slog through getting this packaged and pip installable if you'd welcome the contribution!
This is super neat, restricting logits to valid tokens is a neat approach. Writing an entirely new json streaming parser to support it even more so :D
I'm currently working on https://github.com/iantbutler01/asimov which is what is being called a semi autonomous agent framework these days. For obvious reasons structured output is super important.
I was looking at approaches to help with that and I really like this, but without a permissive license I'm wary of including it in my project because by default no license means all rights reserved here on Github, even though it seems like your intent is for people to make use of this!
Also I'd be happy to slog through getting this packaged and pip installable if you'd welcome the contribution!