Added action type 'Template'

[rnoffke]

Introducing 'Templates' as action type

'Template' is a new special action type like sample, measurement or simulation (id: -96). It does not allow the creation of new objects, which is realized via a new database attribute disable_create_objects. The main purpose of this action type is to encapsulate frequently reused JSON structures as templates.

Possible applications could be a set of standardized generic metadata attributes that one uses time and time again in different actions or more complex structures you also want to use multiple times in different contexts, like a set of tables that represent a chemical reaction for example. Those sets of attributes can now be templated and then included in real actions like samples.

A simple dummy example should help to clarify the implementation and usage.

In our example we have created two template actions:

1. Chosen Educts:

{
  "title": "Educts",
  "type": "object",
  "properties": {
    "name": {
      "title": "Name",
      "type": "text"
    },
    "educt_steps": {
      "title": "Educt - Step",
      "type": "array",
      "style": "table",
      "minItems": 1,
      "maxItems": 3,
      "items": {
        "type": "object",
        "title": "Step",
        "properties": {
          "educt": {
            "title": "Educt",
            "type": "text",
            "choices": [
              "A",
              "B",
              "C"
            ]
          },
          "step": {
            "title": "Step",
            "type": "text",
            "choices": [
              "1",
              "2",
              "3"
            ]
          }
        }
      }
    }
  },
  "required": [
    "name"
  ]
}

2. Reaction X Constraints:

{
  "title": "",
  "type": "object",
  "properties": {
    "name": {
      "title": "Name",
      "type": "text"
    },
    "refSample": {
      "title": "Referenceable Sample",
      "type": "sample"
    },
    "respUser": {
      "title": "Responsible User",
      "type": "user"
    },
    "collaborators": {
      "title": "Collaborators",
      "type": "array",
      "style": "list",
      "items": {
        "title": "Collaborators",
        "type": "user"
      }
    },
    "startDate": {
      "title": "Start Date",
      "type": "datetime"
    }
  },
  "required": [
    "name"
  ],
  "propertyOrder": [
    "name",
    "refSample",
    "collaborators",
    "startDate"
  ]
}

You can now include these templates in another action by either using the visual editor:

Or you can also just insert a empty JSON Dummy that would look like this:

"educts": {
      "title": "Educts",
      "type": "object",
      "template": "34"
    }

The newly created sample action then won't actually only contain this dummy JSON Block, but instead the entire "Chosen Educts" object structure injected in its place. This means the newly created Action, once it is stored in the database, is independent from all used templates. Subsequently all created Objects are also independent, so that the process of updating objects to updated action schemas is completely unaffected.

The only remaining artefact is the otherwise meaningless attribute "template": 34 for the object. This remains in place, so that we can inject the dummy again solely in the context of the action editor (both visual or JSON). That means that once you try to edit this action again, you can still use the visual editor no matter how complex the structure behind the stored templates might be. Even if you use the JSON editor the action remains clear and a lot less cluttered. Once you save your changes to the surrounding structure the template data gets reinjected again.

This has the additional upside to allow small changes to an action via the visual editor for less tech-savvy users.

[FlorianRhiem]

Hey @rnoffke, thank you for the pull request.

This definitely seems useful, but I don't really see why these templates for parts of an action schema should be actions themselves. Actions model processes that generate research data in the form of objects. I can see why an administrator might want to temporarily prevent users from using actions of a given type, so disable_create_objects could be useful, but if an action type is generally not meant for being used and would not make sense if it was used, then why make it an action type in the first place? Wouldn't it be much cleaner to have these action schema templates as their own entities?

[rnoffke]

Hey @FlorianRhiem and @maltedeckers,

thanks for the feedback.

We added the 'Templates' as an action_type to reuse the entrire existing structure of handling actions. In the background templates just are actions and instead of them it's possible to include any action (using the JSON-editor) also. The reason why only template-actions are selectable in the visual editor is to avoid confusing the user and to create a logical seperation. In that sense it would probably be better to further clarify the unique role of templates to the user by creating a completely seperate menubar entry, so that it won't be listed as an action_type explicitly.

[FlorianRhiem]

We added the 'Templates' as an action_type to reuse the entrire existing structure of handling actions. In the background templates just are actions and instead of them it's possible to include any action (using the JSON-editor) also. The reason why only template-actions are selectable in the visual editor is to avoid confusing the user and to create a logical seperation. In that sense it would probably be better to further clarify the unique role of templates to the user by creating a completely seperate menubar entry, so that it won't be listed as an action_type explicitly.

Okay, I can see why it might be useful to include other actions, and if that is possible, then templates being actions is reasonable. Still, in my opinion, there should be a clear distinction in terms of how they appear to the users, e.g. they shouldn't be included in the general actions list, like you already suggested. Perhaps when Show actions of this type in the navbar is checked for this type, Action Templates is shown in the Other section of the navbar?

[rnoffke]

Hey @FlorianRhiem and @maltedeckers,

we changed the 'template feature'.

Now you'll be able to add a 'template-tag' to every action type. By doing so these actions will be listed to include them into other actions as template.

To hide it in the navbar we removed the 'show_in_navbar' tag by default.

Moreover we will not list actions that are not marked as 'show_in_navbar' in the 'actions-list' any more. They'll be only listed if this action type is chosen specifically. We think this is more intuitive if actions that are not listed at the 'actions' dropdown are listed when you click this dropdown.

[FlorianRhiem]

This looks good, but I got a few more suggestions:

• "Allow usage as template" and related texts strongly suggest a connection to the "Use as template" button for actions, which uses an action as a template for another action. One of these features needs to get a different name and as the current "Use as template" for actions behaves just like "Use as template" for objects, I'd suggest that that feature keeps the name. As these new template actions are included in the schema of another action, perhaps it could be named something in that direction? e.g. "Allow including as Schema Template" and "Yes, allow inluding actions of this type as schema templates in other actions" for the action type, "Schema Template" instead of "Template" in the schema editor and "Schema Template" for the default action type? @maltedeckers do you have a suggestion for this?
• This new feature should be included in the documentation on metadata and schemas, once the naming is fixed
• Using the schema editor to create an action with a template property without a selected template only reports that there is no template selected upon submit, and then selects the first one from the dropdown automatically.
• The name is currently automatically removed from the schema template, this should be indicated to the user in some way when editing a schema template.
• With the template actions not shown in the action lists without the usual hiding mechanism that still allows the author to see them, how can a user get back to a template action they created to e.g. update it? And how can other users know what a template action contains if they have no way of seeing them?

Also, please check for code style violations using python3 -m pycodestyle --ignore=E402,E501,W504 sampledb and python3 -m pyflakes sampledb and adjust the pytest tests (currently at least tests/api/server/test_action_types.py::test_get_action_types fails).

[rnoffke]

Hey @FlorianRhiem and @maltedeckers,

we changed the code concerning your annotations.

• The 'Template' action_type changed it's name to 'Schema Template'. Moreover the translations and the related texts changed.
• The 'Schema Template' has '-' as default text values to being abel to edit it using the visual editor.
• Trying to create an action containing an empty 'Schema Template' will cause an error found by the javascript that holds the visual editor.
• Creating an action which actiontype is markes as 'is_template' will cause a notification that the name will not be included into the action schema of actions including this 'Schema Template'.
• Not admin users are able to view actiontypes. They're not able to edit them or look into the current settings. The 'action_template' link will be shown in the navbar for any user. This way the users can see actions from action_types that are not shown in the navbar.

Moreover some buggs were removed (e.g. keys that are only supported in not root objects will not be added to actions including Schema Templates containing these keys) and the code style has been improved.

We will add the documentation if the current version does not point out any other issues.

[maltedeckers]

Hi @rnoffke,

I think showing template actions in the actions list (/actions/) in a separated section might be more obvious than having to access it by the action types list. That way we could hide the action types list from "normal" users again. Maybe accessing action types not shown in the navbar could still be useful, but then a way to hide action types from users completely might be needed (@FlorianRhiem ?).

[maltedeckers]

I get this schema validation error in the attempt to use a simple template containing only a simple text value. Am I using the template feature wrong or is this a bug?

The template:

[maltedeckers]

At least tests/logic/test_action_type_translations.py::test_delete_action_translation seems to fail.

Also tests covering this new feature would be great.

[FlorianRhiem]

I agree that showing action types to users with them unable to do anything isn't great. The basic concepts of SampleDB already need some time to get used to, we shouldn't make it any more difficult for new users. Personally, I think a Schema Templates section in the More category of the navbar might be the cleanest. If they are listed among the actions in the action list, then using them as an action should make sense. If it doesn't, then listing them there will lead to confusion.

[FlorianRhiem]

We've merged the changes, along with a few more improvements for handling nested templates, etc. Could you post on the mailing list to explain this new feature and the ideas behind it?