search

iwoplaza / typed-binary

Describe binary structures with full TypeScript support. Encode and decode into pure JavaScript objects.
https://iwoplaza.github.io/typed-binary/
MIT License
104 stars 3 forks source link

JSON Schema / similar easy tree/array-like object schema use #31

Open jrysana opened 1 month ago

jrysana commented 1 month ago

I'd really love to use a library like this to facilitate the transmission of JSON Schemas - or similar, isomorphic or homomorphic schema description for describing JS objects and other structured data.

For context, right now I'm using UTF-8 encoded JSON Schemas derived from Zod schemas (using modified zod and zod-to-json-schema) to transmit instructions to a language model server that ingests and compiles those schemas into constraints to control inference.

I've noticed when I load in huge schemas (e.g. to describe thousands of separate functions and their parameters) the time on the wire can get pretty bad, so I want a compressed binary representation that could be faster to transmit and ideally also faster to parse/build.

iwoplaza commented 1 month ago

I see! Thanks for opening up the issue. Do you have a set of examples of such large-scale JSON schemas that you can share publicly? Would help in being able to benchmark a potential solution.