n0-computer / beetle

Other
34 stars 15 forks source link

Embeddable IPFS database? #88

Open joepio opened 1 year ago

joepio commented 1 year ago

Hi there, this project looks amazing! I've been looking for a rust IPFS implementation for a while, and was getting worried that rust-ifps was becoming stale. Since two weeks ago, it is officially no longer maintained.

Having said that, I was wondering if iroh will be usable as an embeddable solution. You've already registered a crate name, anyways!

My Usecase

I'm working on a Notion alternative / graph database / knowledge management system called Atomic-Server. It's designed to be highly decentralized, allowing users to re-use schema definitions from other users. However, as it uses HTTP for resolving these schema definitions, the system is reliant on the servers to stay online. Currently, all servers cache all schema definitions. But if a new server joins, and the original host is offline, the HTTP identifiers will not resolve. This is where IPFS comes in! I'd like schema definitions for Properties and Classes to be referred to by IPFS identifiers. But I also want to prevent having any runtime dependencies. I want my users to run the entire application using one single binary. I guess you understand why I'd like to have a rust IPFS crate then! Currently, my implementation is using actix_web, although I've been considering switching to axum (like you are using).

What embeddable iroh might look like

I'm not entirely sure how an embeddable version of iroh would work, but maybe something like this:

fn main() {
   let config = iroh::Config::default();
   // Starts a server, perhaps at some subdomain or a specific path
   let ipfs_instance = iroh::Server::start(config).await;
   // Allow users to write / read from the IPFS universe
   ipfs_instance::add("hello world");
}

Would love to hear your thoughts on this!

b5 commented 1 year ago

apologies for the delay @joepio, we're absolutely planning on supporting a use case like yours. I'm in the process of working up the required changes, and we should have an example that looks a lot like your code sketch above this month.

I'll leave this issue open, and let you know once we can do embedded iroh here!

joepio commented 1 year ago

That's great, thanks @b5! Let me know if I can help with this.

matthiasbeyer commented 1 year ago

I am also interested.

Features and Usecase I am interested in I am interested in building a desktop application which uses IPFS as a backend. The features I am especially interested in are DAG access, gossipsub support (high level, I don't want to care about the details) and storage. I want to be able to push JSON/etc into IPFS and access it via IPLD. IPNS would also be of interest.

Edit: So I've been playing for the last three hours or something like that with the iroh crates and I must say that I am impressed. A lot of stuff is already there, even DagCBor and other DAG formats! But what is dearly missing for me as an application developer is a high level API. I don't want to mess with five different crates (figure just made up here, to get the point across). I want to have one interface where I can instantiate my IPFS node, put text, images, videos, etc into the storage, add a IPLD datastructure for referring between blobs, build a DAG using IPLD and add structured data with IPLD, iterate recursively over that IPLD stuff and so on and so forth.

That said, I have high hopes that this project here will fullfill these needs in the not-so-far future! Please keep up the good work!

conanchen commented 1 year ago

I want the high level APIs : https://github.com/ipfs/js-ipfs/blob/master/docs/core-api/FILES.md