Blobs support

[pfrazee]

Opening this to track the idea.

This is a feature that could live here or at a higher layer, but one thing I found useful in CTZN was support for blobs which could be attached to records. That way you could share things like profile pics and post attachments under the same "logical db."

In CTZN, I implemented this in basically the same way hyperdrive does it - I had a second hypercore which contained the blobs, and then included records in the bee which indexed into the blobs. Hyperbee's metadata has a field for a contentFeed which we could use for this.

The challenges to something like this:

• It breaks from the standard hyperbee URLs a bit, because - wherever you place the blob pointer record - you want that to be able to resolve to the underlying blob. Something like hyper://key/my-record/1/blobs/1 which outputs the binary.
• On record deletes, you need to be sure to delete the attached blobs as well, or else they'll accumulate indefinitely.

An interesting aside -- it would make a lot of sense to gzip blobs when they're stored. Then the API can specify whether to gunzip when the blob is requested. Not only does this save space, but if you don't gunzip server side, the browser can do it on render and you're minimizing bytes over the wire as well as on disk.

[RangerMauve]

Interesting idea. Do you have any dream APIs of how it would work?

I think BSON supports a binary data type which could be useful. It just stores an arbitrary buffer of data. https://bsonspec.org/spec.html

Not sure if there are perf limitations on leaves in hyperbee either.

[RangerMauve]

Gonna close this for now since there didn't seem to be any interest from users.