search

superduper-io / superduper

Superduper: Integrate AI models and machine learning workflows with your database to implement custom AI applications, without moving your data. Including streaming inference, scalable model hosting, training and vector search.
https://superduper.io
Apache License 2.0
4.65k stars 448 forks source link

The serialization and deserialization of Query cannot be shared across different backends. #2410

Open jieguangzhou opened 3 weeks ago

jieguangzhou commented 3 weeks ago

Currently, Query is actually divided into two types: MongoQuery and IbisQuery. When used in a MongoDB scenario, after export, it is recorded as superduper_mongodb, making it impossible to migrate to Ibis.

Our current application heavily depends on Query, which leads to a strong dependency on the backend during application construction and export, making it difficult to establish a unified application standard

https://github.com/superduper-io/superduper/pull/2363#discussion_r1700248212

In the future, all Query classes can be merged into one class, with Query only recording the expression and being independent the type of the database. We can use Databackend to execute the Query

blythed commented 3 weeks ago

Solution: serialize with a base function: superduper.backends.base.parse_query.

jieguangzhou commented 2 weeks ago

2417