search

superduper-io / superduper

Superduper: Build end-to-end AI applications and agent workflows on your existing data infrastructure and preferred tools - without migrating your data.
https://superduper.io
Apache License 2.0
4.81k stars 464 forks source link

[PRE-RELEASE-0.4] Create utilities to avoid using the `predict_id` in queries #2436

Closed blythed closed 1 month ago

blythed commented 2 months ago
  1. db['_outputs__<identifier>__<uuid>'] should work even if user gives db['_outputs__<identifier>'].
  2. db['docs'].select().outputs("<identifier>__<uuid>") should work as db['docs'].select().outputs("<identifier>")

The expected behaviour is that the latest version should be loaded.

TODO: check that these are the only places needed.

jieguangzhou commented 1 month ago

TODO:

  1. As a condition for the filter.
  2. After exporting the application, check whether it is correctly read when reloaded.
jieguangzhou commented 1 month ago

The current adaptation for export and import requires modifications based on some refactored logic. Therefore, this PR/issue is temporarily on hold and will continue after the refactoring is complete.

https://github.com/superduper-io/superduper/tree/feat/predict_id_utils_new

blythed commented 1 month ago

Option 1

When saving a template remove specific uuid from a db[<outputs>].select() query

This is dangerous, since the query is no longer "bound" to a version.

Option 2 (not exclusive)

When querying the db allow users to omit the uuid and fetch the latest one

For example db['_outputs__my-listener'].select()

Option 3

Enable reference to attributes in component.json; for example: ?my_listener.outputs; intelligently infer this information when exporting

This binds the reference, without being explicit. The implementation will require maintaining a cache of components when exporting/ serializing components.