Relay Service Abstraction and Addressing

[ramwan]

Relay handles the networking aspects of automatons. Things that need to be examined are:

• service discovery
• addressing and naming of services and content

Service Discovery

• services provided by automatons to automatons within a network
• services provided by automatons to clients
• permanent services provided by ephemeral automatons

Content and service addressing/naming

• meaningful names of automatons and services
• modular and adaptable naming to accommodate changing specs and requirements

Context

Let there be a service A which needs to talk to a service B to work. A classic example would be a web server and a database.

A <---- message ----> B

We can have a few different structures to consider:

• one-to-one
• one-to-many
• many-to-many

We also have multiple scenarios: (The following may not enumerate all scenarios.)

• The service provider has crashed/shutdown and there are no other instances running
• The service provider has crashed/shutdown and there are other instances running
• The service provider needs to be migrated in the middle of a communication flow
• A service provider needs to be found and contacted (different structures as above)
• A new service provider has just been created and is ready to start serving

Purpose of service abstraction

The service abstraction would make all the issues listed above transparent for the automaton instance and developer. By making the service abstraction meaningful as well, we can allow the service name to be adaptable so changing specs and requirements can be easily accommodated.

What do we need the service abstraction to do?

• allow one service to talk to another without worrying about the underlying networking implementation
• allow the networking implementation underneath to be modular and adaptable to needs
• describe the service in a meaningful manner
  • hopefully be human friendly

[ramwan]

Things to look at:

Serval

https://github.com/MatrixAI/Relay/tree/master/papers/Serval Examination of Serval #9

Scion

Scion may be more to do with how to perform routing and discovery between services. https://github.com/MatrixAI/Relay/tree/master/papers/Scion

DNS/mDNS

mobile networking

[ramwan]

Notes on service discovery after a conversation with @CMCDragonkai

• service discovery in the DNS sense aren't needed in Matrix as when automatons are created, they know the other services they need to communicate with
• resolving of a service address/name can be done at varying levels (eg. by manipulating iptables, user level networking, changing libc)

[ramwan]

Having examined Serval and Mobile IP, I'm going to create a simple scenario to test any design problems we may come across when using abstractions similar to those in Serval. This will test a stateless container responding to pings and then migrating to a different IP address, all transparent to the container sending pings.

[ramwan]

Initial experiment for service abstraction and addressing writeup @ https://github.com/MatrixAI/Relay/blob/master/experiments/stateless_container_ping_migrations/README.txt

The experiment follows roughly the proposal put forward by the Serval architecture albeit only for a single physical host. We have yet to see whether Serval's method of service discovery is necessary or not when scaling to a few physical hosts as well as potentially thousands of physical hosts.

@ the comment I made (2 comments above) about how DNS wasn't necessary, DNS could be a simple enough mechanism to use depending on automaton/service names. Discussed in the writeup.

Personally would say the service abstraction in Serval and mobile IP are all similar in the sense that they provide an extra level of indirection and the experiment shows that it does indeed work. Implementation details though will need to be sorted as discussed in the writeup.

[CMCDragonkai]

Our recent conversation discussed the use of content addressed automatons. Assume some deterministic input (Nix based expression, Content), and hash it, then truncate the hash according to how many bytes is required by IPv6 or IPv4. Make sure to avoid reserved IP addresses (or maybe this doesn't matter within a controlled isolated private network). This is similar to many LibP2P peer id: https://github.com/MatrixAI/hs-libp2p-peer

But basically the addresses also need to be generated relative to the protocol specification. If the session type dictates HTTP, then the address generation should be a dummy HTTP URI (however, there's no need to be random if we can make it deterministic and content addressed).