search

ipfs / notes

IPFS Collaborative Notebook for Research
MIT License
400 stars 33 forks source link

Description language for libp2p services #100

Open void4 opened 8 years ago

void4 commented 8 years ago

Thinking about distributed services in the greater decentralised ecosystem (libp2p, IPFS, Ethereum), I could not find an ontology that takes advantage of the new concepts.

The closest thing I could find is http://noflojs.org/documentation/protocol/ which is a CSP graph based system. It's components have ports that communicate with messages. Specifically, inports (with handlers) and outports are used, which is one of the reasons why I think that a unidirectional (simplex) channel methodology for libp2p could be useful. Maybe it's even possible to go beyond that and work with multiples of expected messages as buffers.

I wrote this in the IRC somewhere: A connection between two nodes could be visualised as a tree of endpoints, whose (multiplexed) stream edges are mirrored for the other node if they are directed.

The format has to be able to describe more than one pattern of interaction. Pub/sub is great but does not cover all use cases.

I currently try to implement the js-ipfs-api and web3.js (Ethereum) on as components on top of NoFlo which could ultimately enable even non programmers to create and interact visually with these systems (files, services, smart contracts). But the real objective is to combine libp2p with noflo at the lowest level, to make things like

possible.

Maybe I'm hallucinating, but I see crazy things happening when this is combined with intelligent routing, reliable data replication and trust systems.

https://groups.google.com/d/msg/flow-based-programming/ziLlktu9qHM/fQuPf6u8CwAJ

So are there any great SDLs out there?

jbenet commented 8 years ago

I have a long standing cautious interest in visual programming interfaces.

A DSL will be simpler for now.

See also: https://github.com/gocircuit/escher and the like.

void4 commented 8 years ago

After many hours of reading I have finally come full circle. And it's beginning to make sense. At first I thought about this issue as one unified problem, but then realised that NoFlo would be only one method of many to create and manage services and the service interface description standard should be independently defined to preserve universality. It was obvious that what was described was a CSP system, but not that the edges connecting the nodes could be different protocols. And this is where libp2p comes into the equation. In the end, the graph is nothing else than a visual representation of programs using libp2p - a set of metadefinitions. Why does a general environment that allows visual manipulation of service interaction not already exist? Because programs need to be transport- and protocol-compatible on more than one endpoint, their interfaces have to be advertised and the distinction between local and remote programs still exists. More here: https://www.reddit.com/r/EthereumProgramming/comments/43zxtd/finding_service_description_ontologies_and/

The "forest of endpoints" does not have to be fully connected if the service/process/node does not require it.

The methods of describing and configuring a service (for example as a graph/ collection of services) should be entirely separated concerns.

A generalised notion of consensus might turn out to be described as identity-relative instead of system-absolute.

I hope that the above is not entirely lunatic and naive (if so, please stop me) and will spend the next weeks to read more about existing open standards and see how they could be applied. Will libp2p use any of them? Will you submit the new definitions to standard bodies such as the IETF?