Update server_options API response

[lorenzo-cavazzi]

The proposed changes for the server options endpoint GET server_options are:

• defaultUrl: since all the strings are accepted, the type should become string (instead of the current enum) and the options should be removed. All the other fields should stay the same. This is an example:

  "defaultUrl": {
    "default": "/lab",
    "displayName": "Default Environment",
    "order": 1,
    "type": "string"
  },

  We discussed also changing the name. Since that is the only field using camel case, and we already allow the snake case equivalent default_url on the renku.ini file, I would change it to default_url also here to keep it coherent everywhere (as well as in the POST /servers API, and wherever is relevant).

• The enum fields should provide options that are objects with label and value instead of just strings representing the values. Wherever possible, the value should be a meaningful number that could be used by the client for getting a closer value when a project default is specified but not listed in the options. E.G. for the Memory, we can have 2 as the value and 2 GB as the label instead of the current 2G.

  "mem_request": {
  "default": "1G",
  "displayName": "Amount of Memory",
  "options": [
      {"label": "1 GB", "value": 1},
      {"label": "2 GB", "value": 2},
  ],
  "order": 3,
  "type": "enum"
  }

[olevski]

@lorenzo-cavazzi I would like to add a few things to this:

• This is more of a question - do you want the default field to use the label or value? I don't really have a preference either way - just thought I would bring it up.
• I would like to add the option to set some fields to be "hidden" so that their defaults are enforced but the selection options are not shown.
• Can we use labels for memory that are compatible with k8s? I.e. Gi or G because it would be nice to show the same units when I return information about the session and what its limit/request in k8s is. If we use GB then I have to convert the value in k8s from G or Gi to GB. K8s does not recognize GB. Or do you think that G/Gi are too confusing for users?
• Can we agree on what the notebooks should expect from the UI when someone is launching a session? Am I going to always get all server options in the request and if anything is missing then I do not launch a session?

So if we add the hidden option then the value of the server_options would be something like this:

"mem_request": {
  "default": "1G",
  "displayName": "Amount of Memory",
  "options": [
    {"label": "1 GB", "value": 1},
    {"label": "2 GB", "value": 2},
  ],
  "order": 3,
  "type": "enum",
  "hidden": false
}

And this hidden field will be mandatory in the values.yaml file.

Also when I say "all server options" I am referring to the list below.

• default_url
• mem_request
• disk_request
• cpu_request
• lfs_auto_fetch
• gpu_request

[lorenzo-cavazzi]

This is more of a question - do you want the default field to use the label or value? I don't really have a preference either way - just thought I would bring it up.

The value, so /lab would work.

I would like to add the option to set some fields to be "hidden" so that their defaults are enforced but the selection options are not shown.

Ok fine, so the UI will ignore all the hidden objects

Can we use labels for memory that are compatible with k8s? I.e. Gi or G because it would be nice to show the same units when I return information about the session and what its limit/request in k8s is. If we use GB then I have to convert the value in k8s from G or Gi to GB. K8s does not recognize GB. Or do you think that G/Gi are too confusing for users?

My idea of labels is a user-friendly name, nothing that would be sent back to a renku-notebooks API. In this case, I would send back the value (i.e. the number of GBs) and the renku-notebooks API could add any required suffix.

Can we agree on what the notebooks should expect from the UI when someone is launching a session? Am I going to always get all server options in the request and if anything is missing then I do not launch a session?

I think all options are currently expected or it fails, right? I would keep it unchanged, so we prevent accidentally starting sessions with unexpected parameters due to a UI error. Do you prefer getting back also the hidden options?

[olevski]

@lorenzo-cavazzi in the latest release of renku-notebooks server options are not expected. You can definitely start a notebook with no options specified or even only with just some of them specified.

The logic is as follows:

• If the server options are not provided at all, then the notebook service goes through the same json object that is returned from the /server_options endpoint and uses all the default values from there.
• If the server options are partially specified in the POST request OR if some server options were not specified in the values.yaml file the notebook service code has hardcoded default values that it falls back to.
• When server options are provided they are validated against the allowed values in the values.yaml file so a session cannot be started with arbitrary values.

What I would really like to do is remove the hardcoded defaults in the notebook service. The unreleased version of the notebook service achieves this by adding a section called serverDefaults to the values.yaml file where defaults for all server options have to be passed. Then it uses this as the ultimate default rather than the hardcoded values. The part of the values.yaml that specifies the server options that are passed to the ui in the server_options endpoint does not require all server options to be included - that is why this new serverDefaults section was added.

What we can do going forward is:

• remove the serverDefaults section from the values.yaml file
• add the hidden property to the existing serverOptions section
• Require all server_options to be specified in the serverOptions section in the values.yaml file. I.e. if a deployment does not use gpus it will still need to include gpu_request in the serverOptions and "lock" its value to zero. The user can decide whether this is hidden or not.
• The UI then does not show any option that is hidden (feel free to do otherwise here I dont have a strong preference).
• As for which options are sent with a POST request after some consideration I am certain that I will always have to both validate and apply defaults regardless of what is sent. This is because even if the UI always does the right thing and provides the right options you could have a request that does not come from the UI and has values that should not be allowed or values that are not specified. So with this in mind the UI can also have full freedom to either not send any server options (in which case all defaults are applied) or just send partially specified server options. And for any values that are sent to the notebook service validation is performed against the server options specified in the values.yaml file.
• This brings me to another point - the UI needs to only allow the users to specify server options for a project that are valid agains the server options provided from the server_options endpoint. I think you already said this would be the case but just wanted to make sure.

[olevski]

And I am in agreement with all other points you made in the last post above.

[lorenzo-cavazzi]

Ok for me it's fine if the API doesn't fail when an option is missing, as long as the default value passed to the UI for every option is also the fallback value applied by the notebook service (this will be the case from your description).

The UI already allows users to choose only values accepted by the server_options. I should double-check if invalid properties (e.g. set manually or through the command line) are already discarded.