multiformats / multicodec

Compact self-describing codecs. Save space by using predefined multicodec tables.
MIT License
334 stars 200 forks source link

Rust implementation built by someone on the libp2p team #337

Open bumblefudge opened 9 months ago

bumblefudge commented 9 months ago

https://github.com/cryptidtech/rust-multicodec

vmx commented 9 months ago

Thanks for posting it. When multiformats and the multicodec table was still young, we had those codec tables in many languages. We then moved away from it. The reason is that almost everyone use any multiformats related would then need to have this as dependency. It surely is less of a deal in a compiled language like Rust (as e.g. compared to JS). Though such dependencies are often a pain if you then have thing like different versions.

We then restructured the multiformats implementations in a way, so that that you can supply them with the multicodec codes that you actually need for your application. That means that if you build something, you should just pick the specific codecs from the multicodec table that you actually need/implement. Applications usually only need a very small set of those, there are only a few rare cases where you want kind of a "catch-all".

bumblefudge commented 9 months ago

Are you recommending that dave's crate be refactored into TWO crates, the parser and the supported codecs, so that projects could just fork and subset the latter when they wanted a lighter build?

vmx commented 9 months ago

I propose not using this crate at all and just define your own constants within your application. In most Rust crates the codes are just integers and not a custom type, so that you don't need such kind of dependencies.

bumblefudge commented 9 months ago

Ah, ok, gotcha. I think the context of this is packaging a general-purpose libp2p library for use-cases beyond IPFS/IPLD, so I presumed it's already a crate that would be maintained over time by some kind of libp2p organization and contains most or all networking-related codecs. So the crate exists for its own reasons.

The orthogonal question is whether the multicodecs folder here on /multiformats/ should point to it as a known, conformant implementation that people can draw inspiration from or cannibalize subsets of? Assuming this box gets checked, of course:

image

vmx commented 9 months ago

If it were me, I'd remove the implementation section from this repo, as the Multicodec table as it exists today, isn't really a concept that should be implemented. It's just a table of identifiers mapping to numbers.

Though as we already have that list and someone opens a PR to add their implementation, I'd merge it.

rvagg commented 9 months ago

If it were me, I'd remove the implementation section from this repo, as the Multicodec table as it exists today, isn't really a concept that should be implemented.

that's a good point actually, and it even still links to js-multicodec which we deprecated; let's remove it eh?