LES: Server load management proposal - detail

[FrankSzendzielarz]

Ref #66 - this adds detail to that general proposal. It is about defining common cross-platform data structures for both hints and configs

Load Management - Policies and protocol

Requirements

• General goal :

  • have the server side load management policy in a common format for all types of client implementation - make this an 'ethereum-scoped' format
  • this format or structure would be common to both server side config, rpc admin calls, and messages sent to clients as hints about the server policy
  • get initial general agreement through discussion here before moving it to EIP
  • apply this to various protocols or aspects of protocols: fast-sync, swarm, whisper
  • allow parts of the server side policy to be requested by clients as 'hints' , so clients can proactively manage the load they incur and avoid server policing
  • avoid unexpected terminations of connections, replace this with frozen connections that respond with 'hints' eg: overloaded, and/or warnings that eviction rules are being approached
  • allow for different types of client, such as paid, free, premium, etc..

• Throttling rules:

  • Rate-limit (token bucket rate limiter)
  • Allow for burst control eg: a rolling limit of 10 requests per second
  • Quota-limit (fixed time period limiter - eg: per hour, day, month)
  • Allow for subscription-like costs or server-wide limits
  • Multiple options for rule scope
  • Per IP:Port
  • Per Node ID
  • For server
  • Per….?
  • Multiple limit units
  • Request count
  • Accumulated message 'weight'
  • ..?
  • API request message types may have an absolute id and corresponding 'weight'. Eg: cap/version/message id : N. The 'weight' is determined by the server admin or indirectly via an automated process governed by the server admin. The 'weight' is akin to the SQL Azure concept DTU, which tries to be a general measure of load, taking into account memory usage, IO and compute.

• Hints and Warnings

  • The general protocol (devp2p) could allow clients to get parts of the throttling and eviction rules as hint messages. These could be supplied on request (initially, or polling for dynamic updates) and optionally on handshake. Which parts would be up to the server.
  • The message format would be common to all implementations because the payload would be parts of the common throttling/eviction rules spec.
  • Reply messages from the server can contain additional hint payload, such as :
    • Per client rate limiter status (token bucket depletion)
    • ..?
  • When a client exceeds the server policies that apply to it, devp2p response messages may include:
    • Rate limit exceeded (equivalent of http 429)
    • Quota exceeded
    • Server temporarily unavailable (equivalent of http 503) - server capacity exceeded due to temporary degradation of resources for the server

• Eviction rules

  • Allow admins to specify under what conditions a client can be evicted:
  • Ignoring too many rate or quota warnings
  • Blowing a rate limiter (0 warnings), as per current LES policy (discouraged)
    • ?
  • These are part of the policy common structure, used as both server config and message to client as a server policy hint

Data Model

On the basis of the above requirements, I would like to propose a data structure that would be common across Ethereum implementations, used to describe the capacity and eviction policies, both as part of server-side configurations, such as configuration files and administative APIs, and also as message body fragments sent to clients as hints.

Example Json

The following example expresses a number of capacity management rules. These would be given to servers to govern the devp2p capabilities, and passed to clients as hints.

To reiterate and clarify: a rate limiter is a token bucket (leaky bucket) limiter aimed at capping burst demand; a quota limiter is aimed at capping overall bandwidth over an absolute unit of time, eg: megabytes per calendar day.

For rate limiters the 'period' describes the refresh rate. So if a rate limiter is 10 calls per 60s, then every 60s the 10 calls will be refreshed (gradually or incrementally, depends on implementation). For quota limiters, every absolute period of time once and once only at the start of the period the limit will be refreshed.

All the following is at this stage fairly rough and intended as a starting point for further refinement.

This example below conveys the following information:

• There is one policy for LES
• The server treats clients as being one of 4 types: free, paid, greylist, banned. Info on the client status should be in some hint or metadata message (TBD)
• The server enforces a number of rate limiters:
  • 10 calls per 60s for free and paid nodes by Node ID
  • 624 total call weight for 60s for free and paid nodes by Node ID
  • 100 cals per 60s by IP
  • etc
• The server enforces a number of quota limiters:
  • 1000 calls or 9000 message-weight (whichever first) per calendar day in server timezone for free clients
  • ten times as much for paid
• There are some eviction rates and quotas, as numbers of violations per...
• The server specifies message weights. The default value is 0.

This would be provided to the server, and the server could also offer clients this in a hint message format.

{
    "capability-policies":[
        {
            "capability":"les",
            "clients": [
                "free",
                "paid",
                "greylist",
                "banned"
            ],
            "rate-limiters":[
                {
                    "calls": 10,
                    "period": 60,
                    "for" : ["free","paid"],
                    "per" : "NodeID"
                },
                {
                    "weight": 624,
                    "period": 60,
                    "for" : ["free","paid"],
                    "per" : "NodeID"
                },
                {
                    "calls": 100,
                    "period": 60,
                    "for" : ["free","paid"],
                    "per" : "IP"
                },
                {
                    "calls": 10000,
                    "period": 1,
                    "for" : ["server"]
                },
                {
                    "calls":1,
                    "period":60,
                    "for" : ["greylist"],
                    "per" : "NodeID"
                },
                {
                    "calls":0,
                    "period":1,
                    "for" : ["banned"],
                    "per" : "NodeID"
                }
            ],
            "quota-limiters" :[
                {
                    "calls": 1000,
                    "weight": 9000,
                    "periodunit": "day",
                    "period":1,
                    "for" : ["free"],
                    "per" : "NodeID"
                },
                {
                    "calls": 100000,
                    "weight": 900000,
                    "periodunit": "month",
                    "period":1,
                    "for" : ["paid"],
                    "per" : "NodeID"
                }
            ],
            "eviction-violation-rates":{
                "violations":10,
                "period":10
            },
            "eviction-violation-quotas":{
                "violation":100,
                "period":1,
                "periodunit":"day"
            },
            "message-weights": [
                {
                    "message": "les/1/getblockheaders",
                    "weight": 12
                },
                {
                    "message": "les/1/getblockbodies",
                    "weight": 62
                },
            ]
        }

    ]
}

Json Schema

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "capability-policies": {
      "type": "array",
      "items": [
        {
          "type": "object",
          "properties": {
            "capability": {
              "type": "string"
            },
            "clients": {
              "type": "array",
              "items": [
                {
                  "type": "string"
                },
                {
                  "type": "string"
                },
                {
                  "type": "string"
                },
                {
                  "type": "string"
                }
              ]
            },
            "rate-limiters": {
              "type": "array",
              "items": [
                {
                  "type": "object",
                  "properties": {
                    "calls": {
                      "type": "integer"
                    },
                    "period": {
                      "type": "integer"
                    },
                    "for": {
                      "type": "array",
                      "items": [
                        {
                          "type": "string"
                        },
                        {
                          "type": "string"
                        }
                      ]
                    },
                    "per": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "calls",
                    "period",
                    "for",
                    "per"
                  ]
                },
                {
                  "type": "object",
                  "properties": {
                    "weight": {
                      "type": "integer"
                    },
                    "period": {
                      "type": "integer"
                    },
                    "for": {
                      "type": "array",
                      "items": [
                        {
                          "type": "string"
                        },
                        {
                          "type": "string"
                        }
                      ]
                    },
                    "per": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "weight",
                    "period",
                    "for",
                    "per"
                  ]
                },
                {
                  "type": "object",
                  "properties": {
                    "calls": {
                      "type": "integer"
                    },
                    "period": {
                      "type": "integer"
                    },
                    "for": {
                      "type": "array",
                      "items": [
                        {
                          "type": "string"
                        },
                        {
                          "type": "string"
                        }
                      ]
                    },
                    "per": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "calls",
                    "period",
                    "for",
                    "per"
                  ]
                },
                {
                  "type": "object",
                  "properties": {
                    "calls": {
                      "type": "integer"
                    },
                    "period": {
                      "type": "integer"
                    },
                    "for": {
                      "type": "array",
                      "items": [
                        {
                          "type": "string"
                        }
                      ]
                    }
                  },
                  "required": [
                    "calls",
                    "period",
                    "for"
                  ]
                },
                {
                  "type": "object",
                  "properties": {
                    "calls": {
                      "type": "integer"
                    },
                    "period": {
                      "type": "integer"
                    },
                    "for": {
                      "type": "array",
                      "items": [
                        {
                          "type": "string"
                        }
                      ]
                    },
                    "per": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "calls",
                    "period",
                    "for",
                    "per"
                  ]
                },
                {
                  "type": "object",
                  "properties": {
                    "calls": {
                      "type": "integer"
                    },
                    "period": {
                      "type": "integer"
                    },
                    "for": {
                      "type": "array",
                      "items": [
                        {
                          "type": "string"
                        }
                      ]
                    },
                    "per": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "calls",
                    "period",
                    "for",
                    "per"
                  ]
                }
              ]
            },
            "quota-limiters": {
              "type": "array",
              "items": [
                {
                  "type": "object",
                  "properties": {
                    "calls": {
                      "type": "integer"
                    },
                    "weight": {
                      "type": "integer"
                    },
                    "periodunit": {
                      "type": "string"
                    },
                    "period": {
                      "type": "integer"
                    },
                    "for": {
                      "type": "array",
                      "items": [
                        {
                          "type": "string"
                        }
                      ]
                    },
                    "per": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "calls",
                    "weight",
                    "periodunit",
                    "period",
                    "for",
                    "per"
                  ]
                },
                {
                  "type": "object",
                  "properties": {
                    "calls": {
                      "type": "integer"
                    },
                    "weight": {
                      "type": "integer"
                    },
                    "periodunit": {
                      "type": "string"
                    },
                    "period": {
                      "type": "integer"
                    },
                    "for": {
                      "type": "array",
                      "items": [
                        {
                          "type": "string"
                        }
                      ]
                    },
                    "per": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "calls",
                    "weight",
                    "periodunit",
                    "period",
                    "for",
                    "per"
                  ]
                }
              ]
            },
            "eviction-violation-rates": {
              "type": "object",
              "properties": {
                "violations": {
                  "type": "integer"
                },
                "period": {
                  "type": "integer"
                }
              },
              "required": [
                "violations",
                "period"
              ]
            },
            "eviction-violation-quotas": {
              "type": "object",
              "properties": {
                "violation": {
                  "type": "integer"
                },
                "period": {
                  "type": "integer"
                },
                "periodunit": {
                  "type": "string"
                }
              },
              "required": [
                "violation",
                "period",
                "periodunit"
              ]
            },
            "message-weights": {
              "type": "array",
              "items": [
                {
                  "type": "object",
                  "properties": {
                    "message": {
                      "type": "string"
                    },
                    "weight": {
                      "type": "integer"
                    }
                  },
                  "required": [
                    "message",
                    "weight"
                  ]
                },
                {
                  "type": "object",
                  "properties": {
                    "message": {
                      "type": "string"
                    },
                    "weight": {
                      "type": "integer"
                    }
                  },
                  "required": [
                    "message",
                    "weight"
                  ]
                }
              ]
            }
          },
          "required": [
            "capability",
            "clients",
            "rate-limiters",
            "quota-limiters",
            "eviction-violation-rates",
            "eviction-violation-quotas",
            "message-weights"
          ]
        }
      ]
    }
  },
  "required": [
    "capability-policies"
  ]
}

[FrankSzendzielarz]

TODO: Add burstiness depth to the rate limiter args.