search

rs-ipfs / rust-ipfs

The InterPlanetary File System (IPFS), implemented in Rust.
Apache License 2.0
1.26k stars 163 forks source link

Ability to derive Serialization to/from IPLD compatible format #417

Open semtexzv opened 4 years ago

semtexzv commented 4 years ago

Is your feature request related to a problem? Please describe. It'd be nice to be able to derive serializing to and from IPLD without having to manually write that code.

Describe the solution you'd like Adopt https://github.com/ipfs-rust/rust-ipld, or develop own capability

Describe alternatives you've considered Using other libraries, developing my own translation layer between structs and IPLD.

Additional context I'm not sure what the situation is with different implementations, but from a perspective of a developer trying to build some custom protocols running side by side with IPFS node the state of things is very confusing. The ipfs-embed is not compatible with other IPFS clients, does not provide pubsub functionality. And the rust-ipfs does not provide tools to work with IPLD in ergonomic way.

Please let me know, what's the state and direction of this feature, I'd be glad to work on it, if needed.

koivunej commented 4 years ago

Hi and thanks for the request!

Could you confirm that this is about the missing ipld derive or what is now https://github.com/ipfs-rust/rust-ipld/tree/master/dag-cbor-derive? It was scoped our during #295 when we needed to pull the dependency in, in order to finally get to #356. The breaking changes in multiformat repos are a bit of an issue for trying to get to compatibility in really short term. As a short term workaround I would assume direct use of rust-ipld features can be used to produce a Box<[u8]> or similar with which we can interop.

Please let me know, what's the state and direction of this feature, I'd be glad to work on it, if needed.

More exploratory work in the IPLD space would be very much welcome! To be honest I must be quite out of date with the latest spec work, but at least for a while it seemed like dag-cbor might be becoming a subset of cbor the wider ecosystem cbor support might have become usable as only single tag (cids) was allowed. With this, perhaps the larger serde ecosystem could be used. As for other ideas I've been floating around related to this: I think striving to make the ipld functionality detached from any of the IO similar to ipfs-unixfs, but this mostly matters for anything multi-block.

I'd also be interested to hear more about your use case.

You also wrote:

[...] does not provide pubsub functionality

Please be aware of https://github.com/rs-ipfs/rust-ipfs/issues/132#issuecomment-696589798 where the gossipsub 1.1 is currently being worked on at https://github.com/libp2p/rust-libp2p/pull/1720.