Distributed App Stack and Abstraction

[cretz]

• I want to build some p2p software, but the abstractions are not well-defined enough if I want to change underlying impls
• Yes there is libp2p and IPFS, but I see that as storage only and needs one level higher of abstraction
• API should have the following
  • Filesystem-style storage
  • DB-style storage (at least CRDTs like Orbit)
  • Pub sub and/or message queuing
  • Identity management (i.e. unique identifiers w/ crypto keys)
  • Team/group management (i.e. asym crypto, secret sharing, etc)
  • Future: remote compute
• Only needs single impl at first, but API must be rigid and documented
• Would like the API in C header for easy adoption (no, this is not a web API), would rather avoid the route other things are taking wrt locally running daemons w/ loopback-based network IPC. Impl can be in Rust.
  • This excludes easy cross-platform Go usage because that stupid platform makes C interop on Windows a pain...but we can explicitly load DLLs or even GetProcAddress on local mod handle
  • The language bindings need too be in the project, not spread around everywhere
• Anonymity is a requirement
  • Would love to require users all be relays, but since initial impl is Tor and Tor makes you be a relay for all or none, relay is opt in

[ghost]

• I would argue that TCP and the IP is very complicated and works really well. The idea of the OSI model can be expanded to include services on a P2P network. Any abstraction over the abstractions you mentioned to do with p2p has to be implemented as a communication protocol. So it should be an RPC system.
  • Check out Retroshare.