Support anonymous Tools

[lcodo]

Add a new Tool property indicating whether the Tool is "anonymous". If so, the Tool object should include only some minimal properties not revealing the tool/workflow identity.

A reference to a non-anonymous" Tool entry should also be included as a not-required property. When an "anonymous" Tool is de-anonymized, (1) a "non-anonymous" Tool document fully describing the tool is set, and (2) the "anonymous" record is updated setting up a reference to the "non-anonymous" analogous document.

[lcodo]

Another way to achieve a dual view is to programmatically control the visibility of the different properties of the Tool object. In this way, there is no need to have two analogous and interconnected instances ("anonymous", "non-anonymous").

• Option A: Add a new property describing the whole visibility profile of the Tool. An option could be to create an array with the list of properties (annotated as JSON paths) to be hidden when the tool is anonymous

• Option B: Add a new property to each Tool property controlling its visibility. An option could be to create a not-required field hide_for_anonymous_tools, of type boolean and and with default value false.

[redmitry]

If the Tool's visibility is the only concern, Option B is much simpler to implement. Option A is preferred if the same must be done for other objects.

[jmfernandez]

If the Tool's visibility is the only concern, Option B is much simpler to implement. Option A is preferred if the same must be done for other objects.

From my point of view, that list of "stealthable" property names / JSON pointers should not be set at the JSON entry level, but associated as metadata or being accessible through a different endpoint. The main point is that, if the user is allowed to fetch an entry, the JSON being returned by API should be valid against its corresponding JSON Schema, which implies that required properties at any level cannot be "hidden".

If the "stealthable" properties are declared at fine grained level (i.e. each entry could have custom stealth rules), or coarse level (the stealth rules are embedded in JSON schema) is an additional level of complexity.

And last, there should be a switch at MongoDB entry hidden metadata level to pass from stealth entry to non-stealth one. I guess the owner of the entry has the last word about when each one of their entries switches between these states.

[jmfernandez]

I have been thinking on this major issue, and after the meeting we had today I have next ideas:

• OpenEBench is currently having Tools entries which are representing both instances of tools which have participated in the different challenges, as well as implementations of Metrics used for assessments. But they are not defining real instances with genuine setups of those tools.
• There is no way to represent anonymized tool instances.

So, I have thought that Tool entries and Tool instance entries should be related, but they should use slightly different identifiers.

• A Tool entry, as it is not representing an instance, does not need several of the current attributes.
• A Tool instance entry has always to be related to a Tool entry, so an _instance_of_ property should exist.
• In order to easily enforce that a Tool instance entry is pointing to a Tool entry, their public ids should follow slightly different patterns:
  1. For instance "^OEBT[0-9]{3}00[A-Z0-9]{5}$" for the Tool entries and "^OEBT[0-9]{3}0i[A-Z0-9]{5}$" for the Tool instance entries.
  2. For instance "^OEBT[0-9]{3}0_[A-Z0-9]{5}$" for the Tool entries and "^OEBT[0-9]{3}0[A-Z0-9]{6}$" for the Tool instance entries.
• A Tool instance entry can be either anonymous or not anonymous, based on an anonymous attribute.

What do you think? The main point I'm afraid of is that it requires reassigning ids to either the tool entries or the tool instances entries,

[jlgelpi]

Actually, we could keep the id patterns, just adding the "instance_of" tag, That would be a kind of internal FK. We could use that to follow from a instance to the entry, and to distinguish entres and instances.

[jmfernandez]

Yes, we can do it without tweaking the id patterns. But the validation logic to avoid loops of a tool which is an instance of another tool, which is an instance of the first one .... should reside on server / API side.

[jmfernandez]

Closed at commit bc48fa7fcba84835b36320a96976135d68220311 (please have a look both at the commit description and the schemas)