Oxen-AI / Oxen

Oxen.ai's core rust library, server, and CLI
https://oxen.ai
Apache License 2.0
176 stars 11 forks source link

Query SQL with duckdb instead of polars. #297

Closed benartuso closed 4 months ago

benartuso commented 4 months ago

Note: a dataframe must be first indexed for querying before it can be queried through either sql or NLP. This will break the current query frontend if merged without frontend changes to accommodate this.

This route is documented in Postman: OxenServer API > data_frame.

Queries and indexes are blocked on dataframes > 1M rows for now, this is configurable.

Note: this needs more tests, which I will write in a follow-up after unblocking Adam on the dataframe editing work. Has been tested manually though

Additionally...I added specific error messages and error types for "this df is not indexed" and "this df is too big" and they're getting swallowed by the hub and just returned as 400 bad request. Think we could really benefit from a mini-sprint on error propagation between our 3 projects - can possibly revisit this after df editing work as well