search

multiformats / multicodec

Compact self-describing codecs. Save space by using predefined multicodec tables.
MIT License
341 stars 204 forks source link

Missing codec: raw string #127

Open najamelan opened 5 years ago

najamelan commented 5 years ago

I think it would be awesome to use this table as a general standard for self describing formats (outside of ipld and ipfs I mean). However a lot of basic things are missing. One I would like to use is just utf8 string. Sometimes, you might just want to send an error message encoded as a string.

I don't know if it would make sense to add all the other string encodings? Personally I try to use utf8 for everything.

It's not clear to me either how nested things can be expressed. gzipped xxx? Maybe compression is something that should be set on the connection rather than the package.

Stebalien commented 5 years ago

It's not clear to me either how nested things can be expressed. gzipped xxx? Maybe compression is something that should be set on the connection rather than the package.

Ideally, compression would be handled at the storage/transport layer (along with "smart" transports that allow sending pre-compressed data without having to re-compress it). However, we'll probably have to support compression at the block/data level anyways.

I think it would be awesome to use this table as a general standard for self describing formats (outside of ipld and ipfs I mean). However a lot of basic things are missing. One I would like to use is just utf8 string. Sometimes, you might just want to send an error message encoded as a string.

Any reason not to just use an IPLD object for this?

najamelan commented 5 years ago

Long story short, personally I have almost no static data to which I want to make addresses, just services. It doesn't seem IPLD is very well suited for this. However I like the idea of self describing formats, of interoperability where possible, and thus it seems like there might as well be a standard for byte encoding formats, like the multicodec table, so why not use it even outside of ipld. This allows us to gradually grow more interoperable software, even though for me today addressing services rather than content is missing for me in libp2p/ipld/ipfs.

Stebalien commented 5 years ago

So, you can do this with IPLD by using the "identity" hash function. We do this to inline small objects into CIDs.

However, I agree we should have a multicodec for this for completeness, regardless. (note: this multicodec would still be a valid IPLD format)

mikeal commented 5 years ago

FYI, we are probably elevating the identity multihash use case for "inline blocks," so this should get easier and be more well supported in IPLD in the future. https://github.com/ipld/js-ipld-stack/issues/11

On Fri, Apr 12, 2019, 10:41 AM Steven Allen notifications@github.com wrote:

So, you can do this with IPLD by using the "identity" hash function. We do this to inline small objects into CIDs.

However, I agree we should have a multicodec for this for completeness, regardless.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/multiformats/multicodec/issues/127#issuecomment-482661422, or mute the thread https://github.com/notifications/unsubscribe-auth/AAACQ1q6qSpYUm2W3R-Zh4kgP584Db7Oks5vgMU6gaJpZM4coDli .