search

n0-computer / iroh

A toolkit for building distributed applications
https://iroh.computer
Apache License 2.0
2.1k stars 137 forks source link

IPFS compatibility, ipfs:// schema support #954

Open joepio opened 1 year ago

joepio commented 1 year ago

Hi there!

First off: really cool project. But I'm not entirely sure if I understand it. Some time ago, I was hoping for an embeddable IPFS implementation that would help me deal with 404-resilience for my project (Atomic Data). The existing Rust IPFS implementations unfortunately stagnated, so I was really glad you started working on iroh!

I just ran iroh on my machine cargo run provide ./src, and that seemed to work well and return a hash of all files, but I kind of expected it to return a resolvable ipfs address for this folder. Where is the ipfs://{CID} URL?

Maybe this project does not (yet?) support IPFS urls, but then I don't really get why it mentions IPFS.

Anyways, awesome to see you guys are making lightweight CID based P2P storage possible!

emmyoh commented 1 year ago

Made a discussion post (#955) about this. Iroh as it was pre-0.3.0 supported grabbing data using IPFS hashes but, AFAIK, that is no longer possible due to breaking changes made since then. I could be wrong?

joepio commented 1 year ago

If it does not support the IPFS schema, that's fine, but I think then a couple of things would need to happen:

b5 commented 1 year ago

Apologies for the delay on this.

I just ran iroh on my machine cargo run provide ./src, and that seemed to work well and return a hash of all files, but I kind of expected it to return a resolvable ipfs address for this folder. Where is the ipfs://{CID} URL?

We aren't doing a good enough job managing expectations here, hopefully this helps:

You can't currently use ipfs:// URI resolution because most of the time ipfs:// URLS are resolved with kubo, which doesn't currently support the data structures Iroh produces. In the coming months Iroh will have a version of a gateway that will let you resolve the hashes iroh produces over HTTP.

Once we stabilize the iroh spec, we'll work on contributing support to kubo. If we can land support there you'll be able to resolve iroh hashes at https://ipfs.io. as for ipfs:// URIs, The exact definition of what ipfs:// means is still being nailed down, even out side of any iroh interop discussion. Once the dust settles there we can give you a firm answer on weather iroh hashes will work with ipfs:// URIs specifically.

We call Iroh IPFS because we're trying to iterate on an improved design of IPFS, not an IPFS-like-thing. Iroh is fully content-addressed, uses CIDs, and is fully peer-2-peer. We've made numerous different design decisions under the hood to build on lessons we've learned after numerous years of work in the community. Long term, we want to end in a place that is both performant and compatible, it's just going to take time.

Hope that helps!

joepio commented 1 year ago

Thanks for the clarification, @b5!

If I'm understanding this correctly, you're saying that the ipfs: schema is currently not entirely specified (still being nailed down), but your current implementation of iroh is incompatible with IPFS's current implementation (kubo).

In that case, I'd still think that it might be better to have an iroh: schema that is fully compatible with your implementation - even if in the long run, it works the same as ipfs:. Here's a few arguments in favor of using an iroh:// scheme:

b5 commented 1 year ago

If I'm understanding this correctly, you're saying that the ipfs: schema is currently not entirely specified (still being nailed down), but your current implementation of iroh is incompatible with IPFS's current implementation (kubo).

Yep. I think that's totally right. We'd like it to be compatible one day, but I don't want to promise you something we haven't figured out how to deliver.

I'd still think that it might be better to have an iroh: schema that is fully compatible with your implementation - even if in the long run, it works the same as ipfs:

My biggest concern here is further exacerbating my fumbling on messaging that we see iroh as an IPFS system. Adding an iroh scheme runs the risk of driving kubo & iroh farther apart, where long term we'd very much want both IPFS to converge.

But, I Hear you loud & clear on this, and I think your point has merit. If we look at more targeted use cases similar to github:// URLs that open the github desktop application from the browser, I think we should absolutely reserve the iroh scheme for a similar highly-targeted use case of telling an application "you can use the iroh protocol for this".

But I'm concerned about folks being confused about when to use ipfs and when to use iroh URIs, which is the exact confusion you've run into here, but having a URI would kinda cement a divide that shouldn't need to be there. This is really a thing we should talk about with other IPFS implementations & nail down an answer on. If we can at all get support for iroh hash resolution in the ipfs schema spec, it would be way more useful for the URI-dissemination use cases we see, like the common case of IPFS references from blockchains.

I'll put some time into specifying what an iroh scheme might entail in the coming weeks so we all have something concrete to look at, then take that to the IPFS Implementers working group & see what they think about it's inclusion in the ipfs scheme.

Apologies for this being so convoluted. Paving the future in the open across numerous orgs is messy.

joepio commented 1 year ago

Thanks again for taking the time to write such a thorough response, @b5. I applaud your effort in merging both projects and preventing a fork / split. Fully understand your considerations.

b5 commented 1 year ago

Thanks so much @joepio. I'll keep this issue open until we figure out a path forward on ipfs and iroh schemes.