Design POST service

[poszu]

This ticket describes the design of the PoST Service. Requirements for the building blocks and a high level description of the API are defined here. The goal is to operate a PoST service independently of the node and phase the functionality out from the node to this new service. This will be done in multiple steps:

Migrate proof generation to dedicated PoST service

The goal of this step is to move proof generation into the new service. This allows operators to run a dedicated PoST service for proof generation that connects to a non smeshing go-spacemesh node. The node would then use the PoST service for proof generation.

Both the go-spacemesh node and the PoST node share the same smeshing identity; although the PoST node only needs the NodeID / Public Key of the identity.

Requirements

PoST Service:

• A new service is implemented as part of https://github.com/spacemeshos/post-rs
  • Releases of post-rs are extended with this service as an executable for already supported OSes and architectures
  • Running the executable starts a server that tries to connect to a given URL where the node the PoST service is used for is reachable
• When started the PoST service reads configuration from the command line and/or a config file
  • Configuration can specify initialisation parameters: NodeID, Commitment ATX ID, size of PoST, max file size and relevant network parameters (labels per unit, scrypt parameters, etc.)
  • Initialisation parameters do not override existing data, i.e. if initialisation already started postdata_metadata.json takes precedence (and operator is informed about deviations from config via logs)
  • Initialisation via the service itself will be implemented in a future change (TODO mafa: create ticket(s))
  • The configuration should include information about which node to connect to: URL & authentication info
  • The service should be able to identify itself to a node (so a node only allows connections from known PoST services)

go-spacemesh:

• go-spacemesh needs to be extended to allow connections from PoST services
  • initially only from one instance which will use the same identity / Node ID as the node but eventually from multiple
  • the configuration of node is extended to include certificates from trusted PoST services - only PoST services with trusted credentials are allowed to connect to the node
• When a node is started in standalone mode (i.e. with --start-smeshing) - via smapp or if an operator doesn't want to run PoST on a dedicated host - it starts an instance of the PoST service.
  • The configuration for the service is provided from the nodes configuration in the SMESHING section. Authentication can be omitted in this case.
  • The service is started after the node finishes initialisation and the node will start smeshing as soon as the service has connected to it.
• When a node is started without --start-smeshing it does not initialise or start a PoST service. Instead it waits for such a service to establish an authenticated connection to it and then uses its API for proof generation.

Run a go-spacemesh node with multiple identities

After proof generation via a dedicated PoST service is implemented, the node shall be extended to allow connections from multiple PoST services. Every of these services represents their own identity and the node needs to check when connection is established which service belongs to which identity they manage

• Authentication details need to be added to the node configuration for every PoST service that can connect to it.
• Running a node with multiple identities needs to be checked for possible optimisations
  • e.g. running 10 identities should not require 10x the network traffic compared to using only one identity

Port initialisation to dedicated PoST service

WiP - lower priority; bullet points:

• initialisation in standalone mode should also be done via the PoST service
• functionality from postcli should be integrated into PoST service and postcli either updated to connect to it or smcli extended for that use case

[poszu]

@fasmat

• Configuration can specify initialisation parameters: NodeID, Commitment ATX ID, size of PoST, max file size and relevant network parameters (labels per unit, scrypt parameters, etc.)
• Initialisation parameters do not override existing data, i.e. if initialisation already started postdata_metadata.json takes precedence (and operator is informed about deviations from config via logs)

How about making the initialization parameters optional and:

• read them from postdata_metadata.json if not provided,
• fail if not provided and postdata_metadata.json doesn't exist,
• fail if provided and doesn't match postdata_metadata.json (quit with an error instead of printing a warning)

[fasmat]

How about making the initialization parameters optional and:

• read them from postdata_metadata.json if not provided,
• fail if not provided and postdata_metadata.json doesn't exist,
• fail if provided and doesn't match postdata_metadata.json (quit with an error instead of printing a warning)

Sounds like a reasonable implementation to me 👍