[FEATURE] Extend field level security

[sandervandegeijn]

Is your feature request related to a problem? Field level security is great and essential to our integration infrastructure. Opensearch is connected to our Enterprise Service Bus to consume messages so they can be exposed through an API. A lot of the data is sensitive (personal, GDPR), so FLS provides essential functionality to filter the data a consumer of the API's can consume.

This week we ran into a problem where we came short with field level security. Consider this document:

"_source": {
    "identifier": [
      {
        "identifierCode": "AAAAAA",
        "identifierType": "Accountname"
      },
      {
        "identifierCode": "000000",
        "identifierType": "AccountIdentifier2"
      },
      {
        "identifierCode": "bla@domain.com",
        "identifierType": "userPrincipleName"
      },
      {
        "identifierCode": "blablabla@domain.com",
        "identifierType": "eduPersonPrincipalName"
      }
    ],

If we whitelist the identifier field, it will expose everything under that. What we need is a way to only serve identifierType X and filter out the rest.

What solution would you like? Extension on the field level security to further filter the data

What alternatives have you considered? Different data modelling, but it's not always possible. Implement a filtering proxy but then we would split the FLS-configuration over 2 solutions which is architecturally undesirable.

Do you have any additional context? Add any other context or screenshots about the feature request here.

[stephen-crawford]

[Triage] Hi @ict-one-nl, thank you for filing this issue. To clarify, it sounds like you are looking for a way to "slice" the fields to a more granular level. Is that correct? At the moment, there are some concerns with the integrity of returning sub-fields which have been filtered. Could you provide more context on the implementation you would like to see?

Edit: Pending your findings :)

[peternied]

Capturing part of the discussion during the triage meeting. I think we should support any filtering off of the mapping definition of the index. See Index Mapping for how to look these up on an index.

If we did want to support additional filtering / logic I think it would be better supported via a different implementation such as View, see the following generic proposal

• https://github.com/opensearch-project/OpenSearch/issues/6181

[sandervandegeijn]

Tip from Andrej at the triage meeting: look into using nested fields. Will investigate.

[sandervandegeijn]

Hmm, I tried a sample setup based on Andrej's suggestion:

PUT nestedtest
{
  "mappings": {
    "properties": {
      "user": {
        "type": "nested",
        "include_in_parent": true
      }
    }
  }
}

PUT nestedtest/_doc/1
{
  "group" : "fans",
  "user" : [ 
    {
      "first" : "John",
      "last" :  "Smith"
    },
    {
      "first" : "Alice",
      "last" :  "White"
    }
  ]
}

I'm not able to provide a FLS entry that would only allow me the entry under user where the name is John and not Alice. Can someone point me in the right direction? :)

To elaborate; sometimes we don't have control over what data is provided, but we need to filter out sensitive data. In the case of the first post one of the identifiers was considered sensitive data. We would like to filter the output so that only the insensitive data will be returned.