lock service - design

[cbrinley]

design initial layout for lock service

[cbrinley]

Initial attempt at defining interface to this service and relevant documentation. This may need to be refined. Idea behind parameter passing syntax was to allow easy conversion of this service to a restful interface. This may be a missunderstanding on my part of how droned parses REST input if this syntax is not valid.

Context also has not been fully documented as i am not sure at this phase which variables will be "built in"

[cbrinley]

This service provides inter node locking mechanism. Nodes should use droneblaster command line or relevant API to interact with this service.

Usage: droneblaster locks [] Commands: exists,create,delete,acquire,release,context,admin

Exists: Checks to see if a given lock exists this command when executed from API returns True if the lock exists False otherwise. When used via droneblaster this command will return 0 if the lock exists, 1 otherwise. This can be modified by passing true_value= and/or false_value= as an argument to exists.

----examples---- Example: droneblaster locks exists foo true_value=1&false_value=0

----reference---- the variable specified here only have value for the duration of this call. To permanently set these value use them during the create call or with the context call.

• true_value = the value to return if this lock exists. default is 0
• false_value = the value to return if this lock does not exist. default is 1

  ---

Create: Creates a new lock if it doesn't already exist. By default this will also acquire the lock and return its TOKEN. This can be modified by sending acquired=False to the create call. Any context variable may be submitted with create.

----examples---- Example: droneblaster locks create foo #(returns TOKEN) Example: droneblaster locks create foo acquired=False&resources=3

----reference---- Any context variable can be set via the create call. The variables that are most common to the create scenario are defined here for convenience.

• acquired = by default a lock will be acquired on create. This can be negated by using acquired=False in the create call.
• resources = the number of resources associated with this lock. By default only one resource is created for this lock. This effectively the same as a mutex. When specifying more than one resource the lock is effectively a semaphore.
• admin_only = This variable means that once this lock is created only the admin token can modify or delete this lock aside from standard acquire or release Default is false.
• TOKEN = an alpha-numeric string representing your claim to this lock resource.

  ---

Delete: Removes a lock from the system. New calls to acquire will result in an error until the lock is re-created. A lock token must be passed with this call to delete the token. The admin interface can be used to delete locks without a token. Take care when doing this. In flight coordinated operations could get in a bad state.

----examples---- Example: droneblaster locks delete foo token=TOKEN

----reference---- The delete call requires the current holder of the token or admin token. If admin_only was set to true when this lock was created only the admin may delete.

• token = token to use to delete this lock.

  ---

Acquire: Attempts to obtain a resource associated with this lock. If this lock supports more than one resource it will be decremented from the available resources. The caller can choose to wait on a resource to become available before returning from the call.

----examples---- Example: droneblaster locks acquire foo #(returns TOKEN) Example: droneblaster locks acquire foo wait=10s&resources=2

----reference----

• wait = how long to wait in for this lock to become free. If time expires before lock becomes available call returns with false_value exit code. this parameter supports (s=seconds,m=minutes,h=hours,d=days)
• resources = how many resources to claim from this lock. the TOKEN returned will represent all resources requested and they must be released atomically.
• true_value = same as exists call. the return code to use for success.
• false_value = same as exists call. the return code to use for failure.

  ---

Release: Releases a held lock. This simple command has only one form shown below. Each call to release must include the token returned from "create" or "acquire" in order to successfully release the lock. This can be overriden via the admin interface.

----examples---- Example: droneblaster locks release foo token=TOKEN

----reference---- the token used to release a lock has an implied number of resource associated with it based on the acquire statement. These resources will all be released at the same time. resources = if using an admin token the number of resources to release can be specified.

---

Context: Context returns or updates values stored in the context of a given lock. Some of these values are "built in" and the rest are custom to a given lock or scenario.

----examples---- Example: droneblaster locks context foo get_values=resources,create_time Example: droneblaster locks context foo resources=2&token=TOKEN

----reference---- context can be used to view or update variables associated with a lock. At minimum a lock must be held by the caller with a valid token to adjust variables. Optionally if admin_only was specified during the create process only an admin token my modify variables.

---

Admin: admin interface can be used to override default behavior. Admin actions are defined in the configuration. See configuration example for how to use these. Token use can be restricted to on a number of criteria: source IP public Key used actions allowed

----examples---- Example: droneblaster locks admin foo get_token=True Example: droneblaster locks admin foo generate_token=True&get_token=True Example: droneblaster locks admin foo list_commands=True Example: droneblaster locks admin foo invalidate_token=TOKEN Example: droneblaster locks admin foo list_tokens=True Example: droneblaster locks admin foo pause_token=TOKEN Example: droneblaster locks admin foo resume_token=TOKEN

---reference----

• get_token = This displays an admin token. This can only be executed on nodes that have been registered in the config and using the droneblaster key registered in the config.
• generate_token = if set to True a new admin token is generated for the lock named in this command. Again this is only valid from stations that have rights to run this command and have provided the correct keys.
• list_commands = lists the commands available to run on this station. The list_comands right must be allowed to use this command.
• invalidate_token = Invalidates the TOKEN passed as a parameter. This can be used to revoke access to a subset of users of this lock.
• list_tokens = If True this will list all tokens for a given lock. Admin may use this to selectively revoke certain tokens as needed.
• pause_token = Temporarily revokes the token specified. Similar to invalidate except this can be undone. See resume_token.
• resume_token = Makes a temporarily revoked token (via pause_token) valid again.

  ---

Scenario Examples: ----scenario 1---- node 1 wants to update a database as part of an upgrade node 2 wants to start a webapp as part of the same upgrade node 2 should not be able to start webapp until DB update is complete

simplest usage: node1> TOKEN=$(droneblaster locks acquire upgrades.db) node1> ./do_work.sh node1> droneblaster locks release upgrades.db $TOKEN

node2> TOKEN=$(droneblaster locks acquire upgrades.db timeout=3600) if [ $? -eq 0 ]; then ./start_webapp.sh droneblaster release upgrades.db $TOKEN fi

----scenario 2----
more advanced usage, same basic scenario as 1: node2> TOKEN=$(droneblaster locks acquire upgrades.db wait=3h) if [ $? -ne 0 ]; then echo "error"; exit 1; fi JSON_DATA=$(droneblaster locks context upgrades.db get_values=last_owner,release_time) RELEASE_TIME=$(json.sh get release_time) NOW=$(date +%s) AGE=$(expr $NOW - $RELEASE_TIME) if [ $AGE -gt 600 ]; then echo "lock too old" droneblaster locks release upgrades.db $TOKEN exit 1 fi echo "$(json.sh get last_owner) has completed its work. Restarting webapp." ./start_webapp.sh droneblaster locks release upgrades.db $TOKEN

explanation: usage 1 In the first example we simply acquire the lock on the DB server, do work, and release it when done. notice we must provide a TOKEN, which is returned from acquire to release this lock. This is because without a TOKEN any rogue system could potentially release the lock. Note TOKEN is not sent over an encrypted channel but is a authenticated channel. You should not use locking service where security or privacy is a key element of this transaction. A nosey neighbor could potentiall steal the token and release the lock.

On the webapp server we do the same, acquire the lock, do work and release. This example assumes the webapp actions are being executed after the DB server has already acquired the lock. This is obviously

usage 2 in this second usage example everything stays the same on the db server. on the webapp however we have decided to check the time when this lock was last released and by who. If the lock was released too long ago it could indicate we are using an old lock so we bail for safety. we also ask who released the lock for our output logging. None of this is required and can all be chosen based on the individual need.

Configuration

• SERVICE: &locks SERVICENAME: locks COMMAND_GROUP: &commands1
  • list_commands
  • get_token SOURCE_GROUP: &sources1
  • 192.168.200.23/32
  • 10.45.125.0/24 KEY_GROUP: &keys1
  • /usr/keys/foo.public.key
  • /home/admin/bar.public/key ADMIN_POLICIES:
  • POLICY: &policy1
    • COMMANDS: &commands1
    • SOURCES: &sources1
    • KEYS: &keys1

[cbrinley]

Configuration got jacked up, tired of tring to figure out syntax.... examples don't seem to to work. hopefully this will make it somewhat evident indent level 0: SERVICE indent level 1: SERVICENAME,COMMAND_GROUP,SOURCE_GROUP,KEY_GROUP,ADMIN_POLICIES indent level 2: POLICY indent level 3: COMMANDS,SOURCES,KEYS