[WIP] RFC: Daemon

[FintanH]

Rendered

[FintanH]

What is the scope of the daemon process? Is it per-profile? Per logged-in user? Per frontend application?

Per-profile could be an interesting route, because that means we could easily switch between profiles by stopping one profile and starting up the other - or if they're both already running then just directing requests towards the correct one. This could be controlled by a profile variable -- similar to git's namespacing.

Is per logged-in user different to per-profile? I'm trying to picture what this means but I'm pulling a blank.

Per frontend application seems like the most "dangerous" when it comes to sharing resources. If two frontend applications are using the storage I'd worry that we would run into weird races more often than not.

My thinking is that a daemon process should be linked to one-and-only-one storage in general so as to avoid race conditions for accessing the storage. Following this thinking, my preference would be the per-profile process, starting with our trivial 1-process case since we're limiting to one profile for now.

If several frontend applications share a single instance, and thus the daemon is what it says on the tin: a long-running process, is it a good idea to allow bypassing it in addition? There are numerous potential concurrency pitfalls, not least associated with the profile switching proposed here, so it seems easier overall to funnel all operations through that single instance. More efficient RPC/IPC mechanisms than HTTP/1.1 could be devised for efficiency.

Given the previous point, would the daemon not also need to expose code browsing?

When you say "bypassing" you're referring to something like surf/source being able to access the underlying storage?

Ya, as I mentioned above, I'd also like to avoid concurrency pitfalls as much as possible. So I think you're right in that we need to figure out how we go about this. I imagine we're getting away with it at the moment as surf is essentially doing read-only operations, but what's stopping any process from accessing the underlying storage?

I suppose to prevent other applications from bypassing us we would need to provide a rich API for the storage and require further RFCs for extending it?

We would also need to envelop the source browsing into the radicle-link super-mono-repo -- I would imagine? I would think this needs another plan for doing so. I see some benefits, like surf getting access to git-ext =]

Regarding RPC/IPCs, I think this would be useful/good too. To be honest, it would be nice to define the procedures in something like gRPC or Thrift to make it easy to spin up clients. We gravitate towards HTTP here to make the lives of radicle-upstream easier. Maybe @geigerzaehler and @rudolfs would be interested in this point.

What’s the API for starting, stopping, and inspecting the p2p stack? It surely doesn’t have to be running all the time, or does it?

Good point, this is something that I missed. These would be great additions to the reactor portion of the work. As well as control of the daemon's process we could also have high-level APIs for inspection of the partial view, e.g.

pub trait Connectivity {
    fn peers(&self) -> impl Iterator<PeerId>;
    fn passive(&self) -> usize;
    fn active(&self) -> usize;
    fn connections(&self) -> usize;
}

This seems nice because then the tracing layer can be built on top of these capabilities. What do others think?

What are the similarities with the seed? What are the differences?

With the ideal of this approach being "decomposition for recomposition", the daemon's components should allow for the definition of many different types of applications. I see a seed as just another variant of an application. For example, one seed sub-protocol would be tracking. It would use the Tracking, Project, and Replication capabilities to define how it tracks new projects coming in. For example:

enum TrackingMode {
    All,
    Peers(BTreeSet<PeerId>),
    Urns(BTreeSet<PeerId>),
}

pub fn track<Seed>(seed: &Seed, mode: TrackingMode, urn: Urn<R>, peer_id: PeerId) -> Result<(), Error>
where Seed: Project + Tracking + Replication,
{
    match mode {
        TrackingMode::All {
            match Project::get(seed, urn, peer_id)? {
                None => {
                    Tracking::track(seed, urn, peer_id)?;
                    Replication::replicate(seed, urn, peer_id)?;
                },
                Some(project) => {
                    log::debug!("already tracking", urn = urn, peer = peer_id);
                }
            }
        },
        TrackingMode::Peers(peers) => {
            if peers.contains(peer_id) {
                /* etc. */
            } else {
                log::debug!("not tracking peer", urn = urn, peer = peer_id);
            }
        },
        TrackingMode::Urns(urns) => /* etc. */
    }
}

So to reiterate, I think that the API should give enough tools for consumers to come with their own applications.

As we discussed on the Lynx call, the reactor would be the home for common sub-protocols that are useful to many applications. However, this is where we need to be opinionated on what's "common", or maybe even just what's "useful". For example, asking for a project from the network would be common for user-facing applications but not seeds. So maybe we split things even further and have sub-reactors :)

Something that could help inspire us here is what functionalities do people already envision?

[geigerzaehler]

I also have a couple of questions

• How are client’s authorized? Can a client just send requests or does it need to authenticate itself first? And how do you intend to gain access to the signing key?

• What are you’re plans for dealing with server side events? Upstream often wants to watch for changes to resources.

We gravitate towards HTTP here to make the lives of radicle-upstream easier. Maybe @geigerzaehler and @rudolfs would be interested in this point.

I’d be happy to not use HTTP. In my experience it’s much easier to create bindings (for both client and server) with other RPC mechanisms.

[kim]

Is per logged-in user different to per-profile?

Yes, I was asking about the process: if it is per-profile, something will need to manage 1..n processes. If we are to support running multiple profiles at the same time, then this would also raise the question of how does an application find the port (or socket path) of the profile it is interested in.

Following this thinking, my preference would be the per-profile process

:+1:

what's stopping any process from

accessing the underlying storage?

Nothing, technically, but we could discourage it if the API satisfies all needs.

sub-protocols

I’m not sure I understand how you envision these. Is it more a code modularity thing, or the ability to actually recompose into your own application instead of the daemon?

I was never a fan of local daemons (like IPFS does), but it looks like we can’t do without because we want to talk to browsers. The consequence is that we lose modularity and composability — that web service just has to provide all necessary primitives. What I would hope for, however, is that it provides those in the most general way possible. So, with a grain of salt: do we really need to bake “domain” concepts (“identity”, “tracking”, “gossip”) into the API? Or could those exist only in the client, and implemented in terms of more composable primitives?

[kim]

@geigerzaehler wrote:

How are client’s authorized? Can a client just send requests or does it need to authenticate itself first? And how do you intend to gain access to the signing key?

I think requests should be authenticated, just because it's too easy to fabricate them.

It is important to understand that the signing key ("device key") is really not intended to be exported -- it shouldn't be used for other purposes (eg. code signing). So, in the "central daemon" model the only process which needs to load the key into memory is ... the central daemon. But where does it get it from?

The no-key-reuse constraint applies to other systems, too (eg. IPFS, SSB, ...), so that's presumably why they just store those keys in the application's storage -- an agent or hsm would be nice, but then users are tempted to weaken key security, eg. by storing the key unencrypted so they don't have to enter a passphrase. Otoh, a seed node needs a non-interactive way to get at the key.

Tbh, I don't really know how to solve this, and neither do others. The IPFS daemon, for example, is still unauthenticated -- because, how do you protect the server half of the authentication? Iow, the problem is: the client needs to prove that it owns the private key, but the server needs to own the private key, too.

What are you’re plans for dealing with server side events? Upstream often wants to watch for changes to resources.

I think that's what the seed currently provides, which the daemon doesn't. But, it's a bit of an ad-hoc thing, so I think it should also be standardised what kinds of events are provided.

I’d be happy to not use HTTP. In my experience it’s much easier to create bindings (for both client and server) with other RPC mechanisms.

Since that's not mutually exclusive, do you have something specific in mind?

[FintanH]

Yes, I was asking about the process: if it is per-profile, something will need to manage 1..n processes. If we are to support running multiple profiles at the same time, then this would also raise the question of how does an application find the port (or socket path) of the profile it is interested in.

Ya, these are good questions. So we currently have segregation of profile storage, that means at the least whatever type of concurrency is going that the processes/threads should talk to only one specific storage.

If we used multiple processes and multiple ports then that would seem like it would get real hairy, real fast. The person using the CLI will have to know where to direct requests and can probably fuck things up -- I know I will.

So my other thought might be that profiles can be multiplexed. We could have something like a HashMap keyed by the profile identitifier -- allowing nicks to be used for UX! -- and two methods whoami for getting the current profile and switch for moving to another profile.

A request to the daemon would look at whoami to select the correct running Peer and perform the request on its particular Storage.

I feel like this could be crazy and there's probably some holes yis can poke, so I'm also open to ideas =D

Nothing, technically, but we could discourage it if the API satisfies all needs.

Ya, we're on the same page so. I guess this necessarily means that surf and source would need to rely on librad. Which means that we're likely going to move them over to radicle-link (history repeats itself). Shall I include this in this RFC?

I’m not sure I understand how you envision these. Is it more a code modularity thing, or the ability to actually recompose into your own application instead of the daemon?

I would say these aren't mutually exclusive. My wish for code modularity is so that when we change internal types that the top-level compositions don't need to change -- PTSD from changing the type of someting in state.rs code and this would ripple throughout the code. And if we provide good modular parts then others can compose their own daemons and we promote some exploration in the space :)

I was never a fan of local daemons (like IPFS does), but it looks like we can’t do without because we want to talk to browsers.

So I'll admit ignorance here, what are other options to local daemons? I'm sure we could provide a way to perform one-shot actions from a CLI, e.g. request a project and wait X amount of time in hope to replicate it, then spin down the stack again. But I feel like a long-running will be the most used case.

The consequence is that we lose modularity and composability — that web service just has to provide all necessary primitives.

Can you expand a bit on this point of losing modularity and composability? I'm not sure I fully follow.

What I would hope for, however, is that it provides those in the most general way possible. So, with a grain of salt: do we really need to bake “domain” concepts (“identity”, “tracking”, “gossip”) into the API? Or could those exist only in the client, and implemented in terms of more composable primitives?

Can you provide an example of what kind of composable primitives you're thinking of? If not providing domain concepts then what kind of concepts are we providing?

[geigerzaehler]

I’d be happy to not use HTTP. In my experience it’s much easier to create bindings (for both client and server) with other RPC mechanisms.

Since that's not mutually exclusive, do you have something specific in mind?

Our options are JSON RPC over HTTP, GraphQL, Thrift or gRPC. What is important to Upstream is that we can create type-safe client code with minimal effort. Ideally a large portion of this should be generated. This excludes JSON RPC because of a lack of standardization especially with regards to streams. GraphQL is not a right fit because I don’t see a need to leverage client-defined query and it’s more complex to implement. Finally, I’d prefer gRPC over Thrift because the ecosystem of gRPC seems much more developed.

Are there any other candidates?

[FintanH]

I've drafted up a second version of the overview. I'd like others to see if it seems like it covers everyone's concerns and that I haven't missed any higher-level goals. I'll start fleshing these goals out in more prose then.

Overview

The main goal of this RFC is to provide a complete picture of what the daemon will look like moving forward. There are a few goals to achieve this and we give an overview in the following.

The foundation of the daemon is to provide a set of composable & modular primitives to build more complex components from and allow us to easily switch backing implementations.

From these foundational components we want to provide more out-of-the-box components that describe higher-level processes that are useful for applications, e.g. seed nodes and [radicle-upstream][upstream].

For applications to make use of the daemon they will need a method for interacting with a running process. Here we propose a set RPC methods that can be used by applications to generate client calls. For developers that wish to focus on their work via the command line, we will provide a CLI for interacting with the daemon.

Since many applications will wish to have a real-time feel, we also formalise a model for event streams.

Since our initial implementation of the radicle-link ecosystem is based on [git][git], we take this opportunity to bake a git-server into the daemon for handling the smart protocol.

For radicle-link developers (and of course other consumers), we want to provide a set of methods that allow us to inspect the p2p stack so that we can easily monitor and debug the daemon.

To ensure the security of any applications that rely on the daemon, we also formalise how the daemon will perform authentication for any methods that require it.

The above are the goals for the daemon's APIs, which leaves us with how the daemon acts as a process. We will also formalise how the daemon runs as a process and how this works with the possibility of using multiple profiles on a device and the API for controlling it.

[kim]

So I'll admit ignorance here, what are other options to local daemons? I'm sure we could provide a way to perform one-shot actions from a CLI, e.g. request a project and wait X amount of time in hope to replicate it, then spin down the stack again. But I feel like a long-running will be the most used case.

I never really considered the webapp UA, because I'm very unlikely to ever use one. In a cli view of the world, the only purpose of a daemon is to run the p2p stack if and when that's desired. There are only very few operations which require a running network stack. Everything else I imagined to just be additional porcelain commands to git (which yields modularity and extensibility for free).

But alas, people want to use a browser, so that's where we are.

Can you expand a bit on this point of losing modularity and composability?

We can either have a webservice which exposes everything one can do, or we can have commands which only do one thing (but operate on the storage directly). A zoo of webservices all messing with the same state is definitely not something I'll be +1'ing.

Can you provide an example of what kind of composable primitives you're thinking

Well, the vast majority of things one can do operates on a "Merkle DAG", right? What's inside the DAG nodes depends on the application. Admittedly, graph traversals are not trivial to implement efficiently over a network boundary, but still something to think about.

[kim]

What is important to Upstream is that we can create type-safe client code with minimal effort.

Yes I agree, although I have yet to find something which is not painful. gRPC specifically is (not for the client, but on the other end). Also Rust support seems to be quite behind.

What is more interesting than IDLs is the question if we can do away with a TCP socket. Is the idea that electron makes two network calls per frontend request (frontend -> IPC socket -> backend -> something -> daemon)?

[FintanH]

I never really considered the webapp UA, because I'm very unlikely to ever use one. In a cli view of the world, the only purpose of a daemon is to run the p2p stack if and when that's desired. There are only very few operations which require a running network stack. Everything else I imagined to just be additional porcelain commands to git (which yields modularity and extensibility for free).

Hmmmm well I would envision that it's possible to choose the longevity of the process. As you mentioned above, there can be a control mechanism to stop the daemon. In a webapp-world, they just spin up the daemon and don't bother stopping it. In a CLI-world you can spin up the daemon, do your business, and stop it.

We can either have a webservice which exposes everything one can do, or we can have commands which only do one thing (but operate on the storage directly). A zoo of webservices all messing with the same state is definitely not something I'll be +1'ing.

I'm not proposing a zoo of web services, but rather a zoo of functionality that one service will want to use. For example, the seed and the user application have a few different needs but are backed by some of the same functionality. So having modular components would allow them to define the needed functionality using that shared surface area. So when I say others can build their own daemons, it's more of a competitive thing.

Well, the vast majority of things one can do operates on a "Merkle DAG", right? What's inside the DAG nodes depends on the application. Admittedly, graph traversals are not trivial to implement efficiently over a network boundary, but still something to think about.

So are we talking about at what granularity we're providing the APIs? In my head a trait like:

pub trait Identity<I> where I: Deserialize {
    type Error;

    fn get(&self, peer: PeerId) -> Result<Option<I>, Self::Error>;
}

is allowing the application to choose what the I is by attempting to parse it. I'm struggling to see how one would make that a lower-level call? Would it be providing git commits to traverse? The verification algorithm?

Sorry if I'm missing the point :/

[kim]

it's possible to choose the longevity of the process

That's not my point, my point is the surface area of the API.

For example, the seed and the user application have a few different needs but are backed by some of the same functionality

That's what I'm questioning: there is not really a difference, except certain functionality one might want to turn off or on depending on context.

I'm struggling to see how one would make that a lower-level call? Would it be providing git commits to traverse?

Yeah, essentially. Perhaps worth taking a look at the ipfs API. It's rather terrible when you want to traverse an object graph, but perhaps that could be improved upon.

[geigerzaehler]

gRPC specifically is (not for the client, but on the other end). Also Rust support seems to be quite behind.

This is what I feared. I haven’t worked with it so I wouldn’t even vouch for ease-of-use on the client. Do you have different preference for the RPC layer?

What is more interesting than IDLs is the question if we can do away with a TCP socket. Is the idea that electron makes two network calls per frontend request (frontend -> IPC socket -> backend -> something -> daemon)?

Our mid-term plan is to run the app from a browser to simplify deployment. The ideal setup would be: frontend -> daemon (via browser HTTP APIs). If the daemon would choose not to provide an API with HTTP transport we would create a “proxy” that would tunnel requests from the frontend. Could you explain why an IPC socket would be preferable over a TCP socket?

[cloudhead]

Some thoughts and opinions from my side (some which have already been expressed by others):

• We should focus on the use-case of 1 daemon process running, interacting with 1 monorepo and N apps/clis talking to it.
• There should only be one writer process (ie. daemon) per monorepo (we can enforce via a lock file for eg.)
• We should consider JSON-RPC for the HTTP API, as it's simple and straightforward, unlike REST
• I'd stay away from any kind of profile switching or statefullness outside of the monorepo. If we want to support multiple profiles, the profile ID should be passed in through every request

[FintanH]

Closing in favour of #696