Open willeppy opened 3 months ago
@onefact should be able to help with this!
I wonder if systems like WASMEdge could help here.
one idea is something like https://github.com/LlamaEdge/LlamaEdge
Can use this to prototype, then do a custom Rust implementation of an encoder-only model. That will solve most use cases. (Or use Gemma if it’s small enough? But llama is a more widely used backbone..)
it would be nice if this supports DSPy as well: https://dspy-docs.vercel.app/docs/deep-dive/modules/program-of-thought
Right now, Texture is built around having a python FastAPI backend that holds the data, executes all queries, and is called by the frontend. This isn't totally necessary, and ties the interface to the python backend making it harder to deploy.
I think a better system would be to take an approach similar to mosaic (https://uwdata.github.io/mosaic/server/) where the backend can be totally in browser with duckdb wasm, etc, OR can make calls to a python server.
Rewrite to be in TS
To start making this, I think would require the following steps to get rid of the python backend
I suspect there's probably some code cleanup that would need to happen to have a consistent API that can be implemented by a backend "client" that is either only in javascript, or could be in Python, etc