hitsave-io / xyz

Monorepo for HitSave
1 stars 0 forks source link

feat: blobs #27

Closed seabo closed 1 year ago

seabo commented 1 year ago

Here is a wip re-implementation of evals, which uses S3 as the backing blob storage.

seabo commented 1 year ago

The latest work on this branch is a load of refactoring which I think makes things a bit cleaner.

The next thing to do is to implement Query for stuff which has a content address to a BLOB. So normally, we would implement Query and this would resolve to a type by getting the data out of the DB. What we need is to say that, in some cases, we are resolving to a type which implements BlobAddress, and so when we get that thing back from the DB, we can use the content address to start pulling the BLOB out of S3.

I think we want a unit struct called AndBlob(T) (or GetBlob, and we rename WithBlob to PutBlob). Then when T: Query, we can have an auto-trait implementation impl Query for GetBlob(T). The implementation first delegates to the Query implementation on T to get the metadata from the DB, then retrieves the BLOB with the content address.

seabo commented 1 year ago

see also discussion at #46