WiP: PMSS integration

[bradley-erickson]

This pull request is a work in progress for integrating PMSS into the Learning Observer platform.

[bradley-erickson]

Some settings in the system do a lot of checking to see if an object exists in the settings. Code similar to "google_oauth" in learning_observer.settings.settings['auth'] appears in a few different places and its not clear how we should handle that within LO or PSS. google_oauth is a key along a nested dictionary here.

A similar issue is handling dictionaries in general. Some pieces on LO expect a dictionary coming back from settings. We can easily change something like

# Currently looks like:
sub_dict = settings.settings['item_that_returns_dict']

# Could look like the following instead
sub_dict = {
    'val1': settings.settings.val1(types=['item_that_returns_dict']),
    # ... the rest of the dictionary items
}
# we build the dictionary in the code when we read in the settings

The most complex use case for settings I've encountered thus far is with the KVS. We have

KVS = KVSRouter(items=learning_observer.settings.settings['kvs'].items())

with settings that look like:

kvs:
  default:
    type: redis_ephemeral
    expiry: 86400
  memoization:
    type: redis_ephemeral
    expiry: 60
  ...

I'm not sure the best way to handle this use case. For now I'll leave it as is.

EDIT: The event_auth goes through a similar process of initializing itself.

    for auth_method in learning_observer.settings.settings['event_auth']:

[bradley-erickson]

Feature flags in the system seem a little messy at the moment. Some pieces of the code want to check if a certain item is available or not in the dictionary of feature flags. For fetching a specific setting, the code uses settings.feature_flag(flag). This code attempts to fetch the appropriate feature flag value.

Under the PSS integration, should we 1) modify the feature_flag method, so we can continue calling flag_setting = settings.feature_flag(flag) OR 2) manually call the items with pss flag_setting = settings.pss_settings.flag(types=['feature_flag'])

I think 1 is the correct option here. The modules should probably follow a similar route. At the time of writing this, I already changed the modules to use the 2 workflow (without initially thinking about 1).

[pmitros]

Feature flags in the system seem a little messy at the moment. Some pieces of the code want to check if a certain item is available or not in the dictionary of feature flags. For fetching a specific setting, the code uses settings.feature_flag(flag). This code attempts to fetch the appropriate feature flag value.

Under the PSS integration, should we

1. modify the `feature_flag` method, so we can continue calling
   `flag_setting = settings.feature_flag(flag)` OR

2. manually call the items with pss
   `flag_setting = settings.pss_settings.flag(types=['feature_flag'])`

I think 1 is the correct option here. The modules should probably follow a similar route. At the time of writing this, I already changed the modules to use the 2 workflow (without initially thinking about 1).

I would agree that we should stay with #1. There could be modified from pointing to creds.yaml to a thin wrapper around however we handle this in pss, and eventually, this might make it into pss as a first-class sort of thing (which might go together with lists and dictionaries more broadly).

[pmitros]

Some settings in the system do a lot of checking to see if an object exists in the settings. Code similar to "google_oauth" in learning_observer.settings.settings['auth'] appears in a few different places and its not clear how we should handle that within LO or PSS. google_oauth is a key along a nested dictionary here.

A similar issue is handling dictionaries in general. Some pieces on LO expect a dictionary coming back from settings. We can easily change something like

# Currently looks like:
sub_dict = settings.settings['item_that_returns_dict']

# Could look like the following instead
sub_dict = {
    'val1': settings.settings.val1(types=['item_that_returns_dict']),
    # ... the rest of the dictionary items
}
# we build the dictionary in the code when we read in the settings

The most complex use case for settings I've encountered thus far is with the KVS. We have

KVS = KVSRouter(items=learning_observer.settings.settings['kvs'].items())

with settings that look like:

kvs:
  default:
    type: redis_ephemeral
    expiry: 86400
  memoization:
    type: redis_ephemeral
    expiry: 60
  ...

I'm not sure the best way to handle this use case. For now I'll leave it as is.

EDIT: The event_auth goes through a similar process of initializing itself.

    for auth_method in learning_observer.settings.settings['event_auth']:

We had walked through several ways this might be handled, but IIRC, I think the strategy we converged on was to leave as-is, with list and dictionary-style settings continuing to live in creds.yaml, and moving over incrementally things which fit into pss to use pss, and finishing the complex pieces in a further PR.

[pmitros]

Notes:

• We could have a way to iterate for every type in a selector with some other type.
• We could have a way to iterate through children. Currently, pss treats auth google and google auth as identical. We could not do that, although we'd lost some niceness.
• We could use the direct child selector auth>google (although we lose the ability to swap order)
• We could use the sibling selector auth+google (where we maintain the ability)
• We could be explicit, and have a list of enabled auth methods. It may make sense to have configurations for ones we don't use.
• We could introduce JSON-style list types: auth { methods: ["google","roster","test"]; }
• We could introduce a simpler list style of some kind
• We could introduce accumulating types, where auth { methods: google; } auth { methods: roster; } results in the selector auth methods returning both google and roster. In css, this would override. Accumulation would be an expansion.

We could do more than one of the above too.

[bradley-erickson]

• [x] PSS needs to be published in some fashion before merging