search

Dan-wanna-M / formatron

Formatron empowers everyone to control the format of language models' output with minimal overhead.
MIT License
163 stars 6 forks source link

Native support for Json Schema #11

Closed Dan-wanna-M closed 2 months ago

Dan-wanna-M commented 3 months ago

Currently, we require users to convert their json schema to pydantic class. However, some users may want to interact with inference engine with HTTP/websocket/some kinds of RPC only. To make such usage easier, we should natively support json schema.

bdashore3 commented 3 months ago

I'd like to implement this in tabbyAPI instead of lm-format-enforcer since formatron is fast and supports more structured generation options out of the box.

Are there any pythonic ways to add JSON schema support using formatron's current state? I'm aware of datamodel-code-generator but I don't believe it has a python API to hook into.

Dan-wanna-M commented 3 months ago

I'd like to implement this in tabbyAPI instead of lm-format-enforcer since formatron is fast and supports more structured generation options out of the box.

Are there any pythonic ways to add JSON schema support using formatron's current state? I'm aware of datamodel-code-generator but I don't believe it has a python API to hook into.

@bdashore3 You can check out how callable_schema and dict_inference work. Essentially you can dynamically create a Schema class that inherits abstract classes in schemas.schema and provides field metainfo. Then, you can use it like a pydantic model. I likely will write the same implementation(whose functionality matches current pydantic model schema) in the next few days as well :)

bdashore3 commented 2 months ago

Thanks for the quick response. I'd be happy to wait for your implementation.

If possible, can you please send an example of creating a dynamic schema class?

Dan-wanna-M commented 2 months ago

Thanks for the quick response. I'd be happy to wait for your implementation.

If possible, can you please send an example of creating a dynamic schema class?

You can check out this file. The key is the two lines

_class = type(f"Mapping_{id(mapping)}", (schemas.schema.Schema,), {"fields": lambda: field_infos})
_class.from_json = classmethod(lambda cls, json_str: json.loads(json_str))

. You may also want to check out Python type() docs.

Dan-wanna-M commented 2 months ago

@bdashore3 The latest version 0.4.0 supports basic json schemas natively. It turns out to be much more complicated that what I have expected. This issue is closed in favor of https://github.com/Dan-wanna-M/formatron/issues/12 where I would later articulate a list of all planned supported advanced features.