UCO currently lacks the ability for the expression of a structured interrogatory mechanism (e.g., query, pattern, etc.) to reduce a set of data to a particular desired scope.
This ability is a critical requirement for the multiple cyber application domains including the security operations domain where it is needed for representing signatures/rules, the cyber threat intel (CTI) domain where it is needed for representing threat indicators. and the risk domain where it is needed for representing specific data queries for the analytic evaluation of particular risk conditions. It is highly likely that it is relevant and needed by other application domains as well.
There is a need to be able to express relationships between such selector patterns and other CDO concepts (e.g., Observations, ObservableObjects, Alerts, Malware, Campaigns, RIskMeasures, RiskMeasurements, etc.)
A conceptual structure is needed that is flexible enough to allow subclassing for various particular pattern expression structures/syntaxes/forms.
Requirements
Requirement 1
Ability to name and textually describe a particular data selector pattern.
Requirement 2
Ability to express relationships (potentially temporally variant) between data selector patterns and other CDO concepts.
Requirement 3
Ability to explicitly express various data selector pattern syntaxes (SPARQL, SQL, YARA, GraphQL, STIX pattern, etc).
Requirement 4
Ability to explicitly express manual data selector patterns as a general textual last resort if appropriate explicit data selector pattern syntax implementation is not available.
Requirement 5
Ability to explicitly specify the type (syntax or structure) of a data selector pattern.
Requirement 6
Ability to specify a particular datasource targeted by a data selector pattern.
Requirement 7
Ability to express a particular execution vector (URL, command line, etc) for a data selector pattern.
Risk / Benefit analysis
Benefits
Ability to express specific data selector patterns to support cyber application domain needs (security operations, cyber threat intelligence, risk, etc.)
Provides general basis for a variety of current and potential future use cases involving data selector patterns.
Provides a flexible and extensible yet consistent approach.
Risks
None
Solution suggestion
add new selector namespace;
add new selector:Selector class as subclass of core:UcoObject;
add new vocabulary:SelectorTypeVocab vocabulary;
add new selector:SelectorPatternFacet class as abstract basis of subclass specialization for various data selector pattern syntaxes/structures/forms;
add new selector:GrphQLPatternFacet class as subclass of selector:SelectorPatternFacet;
add new selector:ManualPatternFacet class as subclass of selector:SelectorPatternFacet;
add new selector:SQLPatternFacet class as subclass of selector:SelectorPatternFacet;
add new selector:YARAPatternFacet class as subclass of selector:SelectorPatternFacet;
add new selector:SPARQLPatternFacet class as subclass of selector:SelectorPatternFacet;
add new selector:STIXPatternFacet class as subclass of selector:SelectorPatternFacet;
add new selector:RegexPatternFacet class as subclass of selector:SelectorPatternFacet;
add new selector:ByteArrayPatternFacet class as subclass of selector:SelectorPatternFacet;
add new selector:URLPatternFacet class as subclass of selector:SelectorPatternFacet;
add new selector:selectorType property;
add new selector:selectorTarget property;
add new selector:selectorExecutionVector property;
add new selector:patternString property as abstract basis of subproperty specialization for various data selector pattern serialized syntaxes;
add new selector:graphQLPattern property as subproperty of selector:patternString;
add new selector:manualPattern property as subproperty of selector:patternString;
add new selector:sqlPattern property as subproperty of selector:patternString;
add new selector:yaraPattern property as subproperty of selector:patternString;
add new selector:sparqlPattern property as subproperty of selector:patternString;
add new selector:stixPattern property as subproperty of selector:patternString;
add new selector:regexPattern property as subproperty of selector:patternString;
add new selector:byteArrayPattern property as subproperty of selector:patternString;
add new selector:urlPattern property as subproperty of selector:patternString;
add new associated property shapes on all new classes
Solution discussion
The proposed solution provides a simple class for consistently, yet flexibly, representing and relating structured interrogatory mechanisms (e.g., query, pattern, etc.) to reduce a set of data to a particular desired scope.
The initial set of proposed pattern formats covers some obvious current choices but is not asserted to be complete. Other formats can be added going forward without breaking any existing structure.
This structure is necessary for the Risk application domain and is currently codified and operationally used (as proposed above) by the draft CDO-Risk application domain ontology (currently underpinning MITRE's Supply Chain Security System of Trust Framework).
The structure provides a mechanism to link and to evaluate particular data sources to evaluate particular risk conditions. It forms a basis for manual, semi-automated, and fully automated evaluation approaches.
The upper portion of the diagram below shows a simple overview of a portion of the CDO-Risk ontology including use of the Selector class.
The lower portion of the diagram shows an example how it is and can be used.
This structure is necessary for the CTI application domain as it captures the core context of what a cyber threat indicator is identifying as relevant. In the CTI-CDO application domain ontology the pattern (Selector) will be separated from the relationship of what it indicates in order to support more flexible and powerful expression and analysis.
The below diagram provides a very simple example of how the proposed Selector class would be used to convey cyber threat indicators. In the diagram, objects in orange are class objects from the CTI-CDO application domain ontology.
This structure is necessary for the security operations domain as it provides a structured mechanism for expressing and relating the core concept of signatures/rules.
The below diagram provides a very simple example of how the proposed Selector class would be used within the security operations domain to convey the deployment of a particular signature to an IDS tool. The DeploySelector class object in the diagram would be a subclass of the DeploySelector (action subclass) class proposed in the Software CP .
Beyond its applicability across numerous application domains it also provides significant value in clearly and consistently expressing and relating such patterns across multiple various pattern formats where certain contexts of tools may call for specific formats but the underlying pattern is the same.
The below diagram provides a simple example of a scenario where a STIX (non-CDO) Indicator is received for detecting a file with a particular hash.
To make use of this STIX indicator within the CDO ecosystem and for various contexts and purposes the following could occur:
Capture the purely json serialized STIX Indictor using a UCO ContentData object
Utilize a fictional but completely possible STIX-CDO translation tool to convert the STIX serialization of the indicator to a CTI-CDO Indicator by breaking the pattern out as a Selector and the indicated context out into an Indicator Relationship to a Malware object
Perhaps it is desired to also express and potentially deploy this pattern in other formats such as YARA or SPARQL
Utilize a fictional but completely possible Pattern Polyglot tool to convert between various pattern formats. In this case from STIX Pattern to YARA and to SPARQL and capture the new pattern formats as separate Selectors
Specify new CTI-CDO Indicators linking each of the new Selectors to the same Malware object that the original STIX Indictor asserted as indicating.
Given the above example and diagram resulting in 3 differently formatted Selectors for the same underlying pattern it would now be possible to deploy these Selectors to different tools across different application domains where such a pattern is relevant.
Within the CTI domain where the indicator originally came from we could express that the STIXPattern Selector is deployed to a CTI-relevant tool such as Trend Vision One from Trend Micro that consumes STIX pattern format.
Within the security operations domain we could express that the YARAPattern Selector is deployed to Suricata, a widely used security operations tool that consumes YARA pattern format.
Within a cross-domain context where CDO data from multiple application domains is aggregated into an overall CDO graph we could express that the SPARQLPattern Selector is deployed to a fictional but completely possible CDO Surveiller tool that runs queries against the graph in the background and alerts on any hits.
The proposed Selector construct also makes pattern-based alerting much easier, clearer, more consistent and more flexible.
The below diagram provides a very simple example of expressing an Alert created by Suricata in a security operations domain context when the specified YARAPattern file hash Selector is detected and it can provides details (as an Observation) of what it saw that it believes matches the pattern.
If you look across the last three diagrams it is easy to see how the proposed Selector not only enables numerous tactical use cases but also supports the flow of data across application domains in a more strategic fashion.
Background
UCO currently lacks the ability for the expression of a structured interrogatory mechanism (e.g., query, pattern, etc.) to reduce a set of data to a particular desired scope. This ability is a critical requirement for the multiple cyber application domains including the security operations domain where it is needed for representing signatures/rules, the cyber threat intel (CTI) domain where it is needed for representing threat indicators. and the risk domain where it is needed for representing specific data queries for the analytic evaluation of particular risk conditions. It is highly likely that it is relevant and needed by other application domains as well.
There is a need to be able to express relationships between such selector patterns and other CDO concepts (e.g., Observations, ObservableObjects, Alerts, Malware, Campaigns, RIskMeasures, RiskMeasurements, etc.)
A conceptual structure is needed that is flexible enough to allow subclassing for various particular pattern expression structures/syntaxes/forms.
Requirements
Requirement 1
Ability to name and textually describe a particular data selector pattern.
Requirement 2
Ability to express relationships (potentially temporally variant) between data selector patterns and other CDO concepts.
Requirement 3
Ability to explicitly express various data selector pattern syntaxes (SPARQL, SQL, YARA, GraphQL, STIX pattern, etc).
Requirement 4
Ability to explicitly express manual data selector patterns as a general textual last resort if appropriate explicit data selector pattern syntax implementation is not available.
Requirement 5
Ability to explicitly specify the type (syntax or structure) of a data selector pattern.
Requirement 6
Ability to specify a particular datasource targeted by a data selector pattern.
Requirement 7
Ability to express a particular execution vector (URL, command line, etc) for a data selector pattern.
Risk / Benefit analysis
Benefits
Risks
None
Solution suggestion
Solution discussion
The proposed solution provides a simple class for consistently, yet flexibly, representing and relating structured interrogatory mechanisms (e.g., query, pattern, etc.) to reduce a set of data to a particular desired scope. The initial set of proposed pattern formats covers some obvious current choices but is not asserted to be complete. Other formats can be added going forward without breaking any existing structure.
This structure is necessary for the Risk application domain and is currently codified and operationally used (as proposed above) by the draft CDO-Risk application domain ontology (currently underpinning MITRE's Supply Chain Security System of Trust Framework). The structure provides a mechanism to link and to evaluate particular data sources to evaluate particular risk conditions. It forms a basis for manual, semi-automated, and fully automated evaluation approaches. The upper portion of the diagram below shows a simple overview of a portion of the CDO-Risk ontology including use of the Selector class. The lower portion of the diagram shows an example how it is and can be used.
This structure is necessary for the CTI application domain as it captures the core context of what a cyber threat indicator is identifying as relevant. In the CTI-CDO application domain ontology the pattern (Selector) will be separated from the relationship of what it indicates in order to support more flexible and powerful expression and analysis. The below diagram provides a very simple example of how the proposed Selector class would be used to convey cyber threat indicators. In the diagram, objects in orange are class objects from the CTI-CDO application domain ontology.
This structure is necessary for the security operations domain as it provides a structured mechanism for expressing and relating the core concept of signatures/rules. The below diagram provides a very simple example of how the proposed Selector class would be used within the security operations domain to convey the deployment of a particular signature to an IDS tool. The DeploySelector class object in the diagram would be a subclass of the DeploySelector (action subclass) class proposed in the Software CP .
Beyond its applicability across numerous application domains it also provides significant value in clearly and consistently expressing and relating such patterns across multiple various pattern formats where certain contexts of tools may call for specific formats but the underlying pattern is the same. The below diagram provides a simple example of a scenario where a STIX (non-CDO) Indicator is received for detecting a file with a particular hash. To make use of this STIX indicator within the CDO ecosystem and for various contexts and purposes the following could occur:
Given the above example and diagram resulting in 3 differently formatted Selectors for the same underlying pattern it would now be possible to deploy these Selectors to different tools across different application domains where such a pattern is relevant. Within the CTI domain where the indicator originally came from we could express that the STIXPattern Selector is deployed to a CTI-relevant tool such as Trend Vision One from Trend Micro that consumes STIX pattern format. Within the security operations domain we could express that the YARAPattern Selector is deployed to Suricata, a widely used security operations tool that consumes YARA pattern format. Within a cross-domain context where CDO data from multiple application domains is aggregated into an overall CDO graph we could express that the SPARQLPattern Selector is deployed to a fictional but completely possible CDO Surveiller tool that runs queries against the graph in the background and alerts on any hits.
The proposed Selector construct also makes pattern-based alerting much easier, clearer, more consistent and more flexible. The below diagram provides a very simple example of expressing an Alert created by Suricata in a security operations domain context when the specified YARAPattern file hash Selector is detected and it can provides details (as an Observation) of what it saw that it believes matches the pattern.
If you look across the last three diagrams it is easy to see how the proposed Selector not only enables numerous tactical use cases but also supports the flow of data across application domains in a more strategic fashion.
Coordination