Node registration proposal

[RaasAhsan]

During its operation, a Gemini node must interact with the Gemini master in order to track cluster health, poll scheduled tasks, etc. Accordingly, the Gemini master will need to have the ability to identify nodes in order to carry its various responsibilities.

This calls for nodes to be assigned or generated some token that uniquely identifies that particular node. It should be distinct from a regular node identifier, in that its only purpose is to authenticate with the master and should never be shared with other nodes. Whenever the node has communication with the master, it will specify the token via an Authorization header. The token should be some randomly-generated payload that nodes store in some directory (/var/lib/gemini/token maybe) for reuse across restarts.

The generated token should be assigned to the node by the master via some registration process. This process should be very simple. On startup, the node will read its local token and authenticate with the server (via the ping possibly, this will also help the node determine if it needs to reregister). If the token does not exist, the node will begin a registration process. It will make a call to the master to generate a token for it and start tracking it as a node. The node should persist its token and start issuing pings immediately.

Thoughts?

[JosephMoniz]

This seems like a pretty solid authentication/identification scheme.

Some things to consider.

• Python ships with sqlite, should we use that as a stable state store for nodes instead of files?
• Should the master hold onto node tokens for a grace period passed expiration? (could prevent unnecessary task rescheduling if the node already has tasks running)

[RaasAhsan]

• sqlite sounds much better and much more flexible for internal storage 👍
• I was originally thinking that the assigned token for a node would remain the same forever, but its a better idea to have to regenerate often. Maybe the node would automatically refresh the token well before expiration? Then we can safely assume that once a token has really expired, the node has died, and all their tasks need to be rescheduled.