Create an architecture diagram

[acruikshank]

Description

We would like to have detailed understanding of the go-filecoin's high level architecture. There is currently no good understanding of what perspective to take on the architecture and what it's rules should be.

We have had the concept of an "API" that delineates go-filecoin's interior from, say, network access or the user interface. "API" is in quotes, because the commands also provide a REST API which is not exactly the same.

Porcelain/Plumbing is a good example of a new guiding principal for architecture, but it's not sufficient to structure all the code or inform all architectural decisions.

This issue provides a place to track artifacts of this discussion on their way to becoming normative.

[acruikshank]

This is very high level with almost no detail. What it documents is that porcelain/plumbing provides and interface to protocols. Storage/Mining/Chain are protocols (there could be others). Protocols provide an interface to the network and storage. The diagram suggests that protocols only talk to the API which isn't entirely true.

      ┌─────────────────────────────────────────────────────────────────┐
      │                                                                 │
      │                       Commands / REST API                       │
      │                                                                 │
      └───────────────────────────────▲─────────────────────────────────┘
                                      │
      ┌─────────────────────────┬─────▼─────────────────────────────────┐
      │        Porcelain        │                                       │
      │                         │  Plumbing                             │
      ├─────────────────────────┘                                       │
      │                                                                 │
      └──────────────▲──────────────────────────▲────────────────▲──────┘
                     │                          │                │
      ┌──────────────▼───────────────┐   ┌──────▼──────┐  ┌──────▼──────┐
      │                              │   │             │  │             │
      │                              │   │   Mining    │  │    Chain    │
      │       Storage Protocol       │   │  Protocol   │  │  Protocol   │
      │                              │   │             │  │             │
      │                              │   │             │  │             │
      └──────▲────────────────▲──────┘   └──────▲──────┘  └──────▲──────┘
             │                │                 │                │
      ┌──────▼──────┐  ┌──────▼──────┐   ┌──────▼────────────────▼──────┐
      │             │  │             │   │                              │
      │ filesystem  │  │   bitswap   │   │ Gossip Sub (blocks/messages) │
      │             │  │             │   │                              │
      └─────────────┘  └─────────────┘   └──────────────────────────────┘

[phritz]

Your sketch looks really similar to what I sketched last night (btw what do you use to draw the diagram?). Note i have commands talking to protocols bc of things like client retrieve piece which we want to be synchronous. also mining start and the like. we could route them through an event system but it would be a little awkward though obviously perfectly possible.

I think there are some canonical use cases we should be considering for potential architectures eg:

• chain syncing a new block, validating it, extending the chain with it, block mining seeing the new blocking and mining a new block and feeding it back into the chain sync logic
• doing a synchronous retrieval client retrieve-piece via cli
• embedding a storage client in a golang app
• how protocols get access to protected resources like a signer or a datastore

A big question in my mind is what kind of front vs backdoor dependencies we want. Front door dependencies are accessed through the plumbing api. Backdoor dependencies are via shared direct dependency, eg the chain sync protocol has-a chain store that is the same (read-only) chain store the plumbing chain calls use. For example

• would we want ~all of message pool and chain store functionality exposed via plumbing? Seems like no to me, meaning that we need a crisp way to differentiate when a backdoor dependency is appropriate or not.
• for an iterator like newheaviesttipset clearly the consumer side is available in plumbing. What about the producer side? Seems more natural though admittedly asymmetrical that the producer side is shared via the backdoor. Similarly, the newblock iterator the chain sync protocol uses does not seem like it should be part of the plumbing interface; both the producer side (networking code and block mining protocol) and consumer side (chain sync protocol) seem like shared backdoor dependencies that aren't part of plumbing.

I can hear an argument that everything that protocols need modulo network is available in plumbing. It makes a certain sense but then it effectively flattens the entire abstraction hierarchy into a single interface inclusive of datastore, signing, various stores, etc etc etc and exposes a lot of stuff in the plumbing api that really nobody should be using eg setting a new heaviest tipset. My gut says that plumbing should expose a set of functionality that is complete for user- and tool-facing features but that is not necessarily complete for protocol implementation. For example the producer side of the heaviest tipset iterators or the interface to write to the chainstore would not be plumbing and instead would form part of a backdoor set of deps shared between plumbing and protocol implementations. (according to this theory, which may be wrong)

Happy to hear disagreement on any or all of this.

[phritz]

notes from conversation with @acruikshank wip:

core apis (misnamed as utils above):

• the source of truth
• stores all chain data, the transitive closure of everything reachable from all valid chain blocks
• chain store, consensus, message pool, blockstore, etc

plumbing:

• the set of public apis required to implement all user-, tool-, and protocol-level features
• plumbing implementation depends on core apis

porcelain:

• convenience compositions of plumbing calls
• depend only on plumbing calls
• ephemeral; lifecycle is the duration of the porcelain call: something calls into it, it does its thing, and then returns

commands:

• depend only on plumbing/porcelain, never on core apis
• implement user- and tool-facing features

protocols:

• persistent; they keep running without active user/tool activity
• drive changes in but do not own core state
• interact with the network
• often own/maintain their own state eg dealstate
• depends on plumbing and porcelain for a lot of functionality, but also core apis for functionality that should not be public eg setting the heaviest tipset

q: sector base, is that core? @acruikshank q: how do commands interact with protocols? @acruikshank

[acruikshank]

New diagram based on comments above:

           ┌──────────┐ ┌───────────┐ ┌──────────┐
           │          │ │Gossip Sub │ │          │                |  | \  /
  Network  │filesystem│ │ (blocks/  │ │ bitswap  │                |  |  \/
           │          │ │ messages) │ │          │                |  |  /\
           └─────▲────┘ └─────▲─────┘ └────▲─────┘                \__/ /  \
                 ├────────────┘            │
           ┌─────▼────┐ ┌───────────┐ ┌────▼─────┐     ┌────────────────────────────┐
           │          │ │           │ │          │     │                            │
Protocols  │ Storage  │ │  Mining   │ │  Chain   │     │    Commands / REST API     │
           │ Protocol │ │ Protocol  │ │ Protocol │     │                            │
           │          │ │           │ │          │     └────────────────────────────┘
           └──────────┘ └───────────┘ └──────────┘                    │
                 │            │             │                         │
                 └──────────┬─┴─────────────┴───────────┐             │
                            ▼                           ▼             ▼
           ┌────────────────────────────────┐ ┌───────────────────┬─────────────────┐
 Internal  │            Core API            │ │     Porcelain     │     Plumbing    │
      API  │                                │ ├───────────────────┘                 │
           └────────────────────────────────┘ └─────────────────────────────────────┘
                            │                                    │
                  ┌─────────┴────┬──────────────┬──────────────┬─┴────────────┐
                  ▼              ▼              ▼              ▼              ▼
           ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐
           │            │ │            │ │            │ │            │ │            │
     Core  │  Message   │ │   Chain    │ │ Processor  │ │   Block    │ │   Wallet   │
           │   Store    │ │   Store    │ │            │ │  Service   │ │            │
           │            │ │            │ │            │ │            │ │            │
           └────────────┘ └────────────┘ └────────────┘ └────────────┘ └────────────┘

[phritz]

@acruikshank awesome this is looking really good. small suggestions/comments:

• should include retrieval in the protocols
• i would replace the specific networks stuff at top with a network box, maybe with eg bitswap etc
• would be even more clear to have the box under ux be like the porcelain/plumbing box with 'cli' the upper left box and the rest filled with rest api, indicating that the cli is on the rest api, but that the rest api can be called directly
• we should keep message pool message pool, its pretty standard even though yes it is a store
• only thing that keeps sticking in my craw is having a core api as a separate thing from the apis of the core components below. conceptually yes there is but i'm not excited about formalizing it into a actual interface. i think protocols having more fine grained direct deps on core components is fine. happy to have you disagree. but also i dont care enough to argue so will go along with whatever if you feel strongly. i can see the argument from a drawing a line as a barrier pov, and am happy to keep the 'core api' as a concept even if it is not a single thing we can point to in the code.
• please review/comment/extend the descriptions in my comment above if need be

[phritz]

@acruikshank can you chime in on the second to last bullet? @anorth is going to use this diagram in a code walk through so we should settle on how we talk about it.

[acruikshank]

           ┌─────────────────────────────────────┐
           │                                     │
  Network  │  network (gosipsub, bitswap, etc.)  │                 | | \/
           │                                     │                 |_| /\
           └─────▲────────────▲────────────▲─────┘
                 │            │            │           ┌────────────────────────────┐
           ┌─────▼────┐ ┌─────▼─────┐ ┌────▼─────┐     │                            │
           │          │ │           │ │          │     │    Commands / REST API     │
Protocols  │ Storage  │ │  Mining   │ │Retrieval │     │                            │
           │ Protocol │ │ Protocol  │ │ Protocol │     └────────────────────────────┘
           │          │ │           │ │          │                    │
           └──────────┘ └───────────┘ └──────────┘                    │
                 │            │             │                         │
                 └──────────┬─┴─────────────┴───────────┐             │
                            ▼                           ▼             ▼
           ┌────────────────────────────────┐ ┌───────────────────┬─────────────────┐
 Internal  │            Core API            │ │     Porcelain     │     Plumbing    │
      API  │                                │ ├───────────────────┘                 │
           └────────────────────────────────┘ └─────────────────────────────────────┘
                            │                                    │
                  ┌─────────┴────┬──────────────┬──────────────┬─┴────────────┐
                  ▼              ▼              ▼              ▼              ▼
           ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐
           │            │ │            │ │            │ │            │ │            │
     Core  │  Message   │ │   Chain    │ │ Processor  │ │   Block    │ │   Wallet   │
           │    Pool    │ │   Store    │ │            │ │  Service   │ │            │
           │            │ │            │ │            │ │            │ │            │
           └────────────┘ └────────────┘ └────────────┘ └────────────┘ └────────────┘

[acruikshank]

@phritz Everything but the second to last comment should be reflected in the new diagram. As for whether there should be a Core API, the point I'm trying to make in the diagram is that there's a boundary between protocols and the things (stores?) at the bottom. Protocols can depend on those things, but not the other way around. Protocols implement processes, and the things at the bottom provide infrastructure to support those processes.

Wherever we can draw a boundary like this it's a good opportunity to think of it like an API (i.e. ask whether it's user friendly and well tested). That said, I don't feel strongly about formalizing the API beyond formalizing the rules that separate protocols from stores.

[phritz]

awesome thanks @acruikshank . @anorth please close this when you include the diagram above in your doc.

[anorth]

Now in CODEWALK.md