IPFS Implementation - Rust

[jbenet]

It would be great to have one.

[nham]

I've made some initial attempts at implementing some of the base libraries in Rust.

• rust-multihash
• rust-base58

[jbenet]

@nham great!

cc @rschulman author of https://github.com/google/rust-multihash -- you guys may want to work together.

I'll be spending a lot of time on https://github.com/ipfs/specs next week. a goal is to synthesize a list of libraries/modules to implement. perhaps that's a good step forward.

i'll also include other people that may be interested in rust impls here.

@nham and @rschulman happy to setup a rust team in the ipfs org and any repos (like http://github.com/ipfs/rust-ipfs) for you if you want. similar to http://github.com/ipfs/py-ipfs and http://github.com/ipfs/go-ipfs

[nham]

@jbenet I would be interested in participating in the Rust implementation, but I am still working my way through the go-ipfs code and the paper.

[chpio]

is there any progress?

[dignifiedquire]

I made some progress with this earlier this year, but we are far from a fully working implementation right now.

[dignifiedquire]

Some more things happened in rust

• https://github.com/multiformats/rust-multiaddr
• https://github.com/multiformats/rust-multihash
• https://github.com/micxjo/rust-ipfs

[kumavis]

wanted: rust-cid

[dignifiedquire]

There is now https://github.com/multiformats/rust-multibase which is ready and published on crates.io

https://github.com/ipld/rust-cid is still WIP

[dignifiedquire]

Published over the last couple of days

• rust-cid
• updated and improved rust-multibase
• updated and improved rust-multihash

[dignifiedquire]

New version of rust-multiaddr with support for

• ipfs
• ws
• wss
• libp2p-webrtc-star
• libp2p-webrtc-direct

and proper error handling released

[Schaeff]

I've just started playing a bit with Rust, if there's anything I can contribute to I'd be interested

[jefft0]

Has there been discussion of building IPFS natively into the (Rust) Servo browser? It could demonstrate tighter integration than possible in the Firefox add-on.

[dignifiedquire]

@jefft0 no discussions but I did think about it. But I haven’t followed up on it yet. The first thinh we would want for that are some good api bindings from rust to the http api, like js-ipfs-api

[matthiasbeyer]

Are there any updates on this issue? I would like to have a IPFS/IPNS/IPLD interface from Rust for one of my project ideas.

[daviddias]

rust-libp2p is coming up! https://github.com/paritytech/rust-libp2p

[atomashpolskiy]

Hi guys! Is there anything up for grabs? :)

[mkazlauskas]

There hasn't been an update to this issue for a while. Could anyone elaborate on what parts we already have and what's still missing?

[Chiiruno]

I would like to know too, I'm very interested in this for a project I want to do.

[Stebalien]

Rust libp2p (now at https://github.com/libp2p/rust-libp2p) is coming along but I'm not aware of any work on rust-ipfs.

[magik6k]

Related issue in rust-libp2p repo: https://github.com/libp2p/rust-libp2p/issues/233

[adamski]

I'm an experienced programmer although I'm new to Rust, I'd be very much up for collaborating on this.

[ebkalderon]

It seems rust-libp2p is advancing nicely. So I understand, if there is to be a Rust implementation of ipfs, would it probably be based on this crate?

[Stebalien]

Probably.

[saresend]

Hey! I've done some work in rust, but am quite new to IPFS. I'd be very interested to help out and I was wondering if this is being actively worked on beyond developing rust-libp2p?

[daviddias]

To everyone interested in creating rust-ipfs, one great way to start is by checking the rust http client libraries for IPFS and testing it against go-ipfs and js-ipfs daemons -- https://github.com/ipfs/ipfs#http-client-libraries --. You will learn a ton by updating them and by doing so, you will create a set of tests in which an rust-ipfs implementation will also have to pass, if the client library and the native implementation end up exposing the same interface (which they should :))

[dvc94ch]

So there is a rust-ipfs implementation that is a month old and a little messy at the moment but should get there quickly. Who should I talk to about moving it into the ipfs org?

• [0] https://github.com/dvc94ch/rust-ipfs

[matthiasbeyer]

Any progress here? I'm working on a project that uses IPFS internally and am currently talking to the IPFS daemon using the rust-ipfs-api crate... I would love to include a rust-native ipfs implementation, depending on whether it is on-par featurewise and compatible...

Who can I talk to? I would also participate on a rust implementation for this, but AFAICS there are several attempts... joining efforts would be (my personal) no. 1 prio here...

[dvc94ch]

It's already bitrotting. I'm working on a new rust-ipld crate, and refactoring the networking code since it currently uses some hacks for mdns that aren't going to be accepted upstream in libp2p. These two tasks will take a while at the current rate, the rust-ipld crate is nearly complete.

[aphelionz]

@dvc94ch Made a PR against your rust-ipfs crate. https://github.com/dvc94ch/rust-ipfs/pull/32

If anybody else wants to weigh on on how to update the last two files to get it to compile and run, that wold be great! I'm still a beginner with Rust but I got pretty far.

[bedax]

New to IPFS, so apologies if I've missed something, but has anyone had any success linking to the go/c implementations from Rust? Having a ipfs-sys crate that handles the linking would be a nice stand-in while the rust-ipfs implementation finishes

[aphelionz]

Happy to report that a substantive effort has begun over at https://github.com/ipfs-rust, with support from PL. Join me, @dvc94ch and @koivunej over there if you please :)

[prataprc]

IPFS is awesome project. Wanted to take a deep dive, with Rust :). Instead of selling should-do-it-in-rust idea, I thought of measuring how-big-is-IPFS-in-go

So, how big is IPFS project ?

go is cool, I can do this:

GOPATH=/home/prataprc/ipfs go get github.com/ipfs/go-ipfs/...

and fetch the entire code base, that is required to build ipfs.

To begin with, ipfs is fetched from 7 repository servers: bazil.org, github.com, go4.org, golang.org, google.golang.org, go.opencensus.io, go.uber.org, other than github.com most of them are common-library.

Let us just stick to github.com. Within github.com we need repositories from 58 users/organisations. Checking them one by one, most of them are common-libraries (or optional-libraries) that are most likely available in rust. Before filtering them out, let us see why do we need them.

clock, statistics, rate-limiting, proquint, hashing, generics (really?), terminal-ui, semver, systemd, man-page, git, data-compression, cryptography, data-store, pretty-print, json-diff, json, protobuf, grpc, http.

And some of them are not so common:

• github.com/cskr/pubsub
• github.com/btcsuite/btcd
• github.com/flynn/noise
• github.com/huin/goupnp
• github.com/jackpal/go-nat-pmp
• github.come/koron/go-ssdp/
• github.com/lucas-clemente/quic-go
• github.com/miekg/dns
• github.com/opentracing/opentracing-go/
• github.com/prometheus/

Now filtering them out, we have the bulk of go-ipfs in 106 repositories across 4 organisation:

• github.com/ipfs, with 46 repositories.
• github.com/ipld, with 3 repositories.
• github.com/libp2p, 47 repositories.
• github.com/multiformats, 10 repositories.

Further, 3 simple commands were used to measure the loc of each organisation.

$ cat `find ./ -name "*.go"` | wc # to gather LoC of all golang code.
$ cat `find ./ -name "*.go" | grep -v test` | wc # to gather LoC of all dev code.
$ cat `find ./ -name "*.go" | grep -v test | grep -v "\.pb"` | wc # exclude *.pb.go

Organisation	n_repos	all-go-code	dev-code	exclude *.pb.go
multiformats	10	12091	6860	6860
ipld	3	33648	25454	25454
libp2p	47	186488	138724	109582
ipfs	46	162513	97854	91947

_{* all above metrics in col{3,4,5} are in lines-of-code (approximate) 1 under libp2p/go-libp2p-asn-util/ around 50K lines of code is ipv6-asn mapping 2 under ipld/go-ipld-prime a majority of ~30K lines of code seem to be auto-generated}

Across all these repositories there are 33791 commits, starting from 2014-01-19 to 2020-10-07 by ~500 authors.

Here is the month-wise commit rate:

Here is the month-wise cummulative commits:

Here is the month-wise active authors:

Given this kind of size and development velocity, is it really possible for rust implementation to catch-up with go implementation of ipfs ? Is there a real need for implementing ipfs in rust ?

[dvc94ch]

First it depends on what your definition is of ipfs. The rust-ipfs project will never be as good or carch up to go-ipfs. They embarked on a mission of playing catch-up and bug parity.

If instead you decide to lead ipfs in new directions, to innovate continuously and to strongly reduce the scope to the essentials a single dev can implement ipfs in a month. Checkout https://github.com/ipfs-rust/ipfs-embed

[dvc94ch]

Actually current ipfs implementations are advanced enough that I think this issue can be closed.