search

OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
6.42k stars 947 forks source link

STIX 2.1 import : Windows registry key value erroneous or lost objects #2574

Open 2xyo opened 3 years ago

2xyo commented 3 years ago

Description

Importing a STIX2.1 bundle with a Windows registry key/value observable returns erroneous or lost objects.

Environment

https://demo.opencti.io/

Reproducible Steps

Import the following bundle on https://demo.opencti.io/dashboard/import 


{
    "type": "bundle",
    "id": "bundle--3ca62449-744e-4cad-96a2-5ff302b289bd",
    "objects": [
        {
            "type": "windows-registry-key",
            "spec_version": "2.1",
            "id": "windows-registry-key--9d60798d-4e3e-5fe4-af8a-0e4986f0f90a",
            "key": "HKEY_LOCAL_MACHINE\\System\\FooOnly\\BarOnly"
        },
        {
            "type": "windows-registry-key",
            "spec_version": "2.1",
            "id": "windows-registry-key--2ba37ae7-2745-5082-9dfd-9486dad41016",
            "key": "hkey_local_machine\\system\\bar\\foo",
            "values": [
                {
                    "name": "first",
                    "data": "qwerty",
                    "data_type": "REG_SZ"
                },
                {
                    "name": "second",
                    "data": "azerty",
                    "data_type": "REG_DWORD"
                }
            ]
        }
    ]
}

Example from https://docs.oasis-open.org/cti/stix/v2.1/cs02/stix-v2.1-cs02.html#_6jiqabgqp2hp

Expected Output / Actual Output

Additional information

The export of the 3 observables from OpenCTI is not STIX 2.1 compliant: 

{
    "type": "bundle",
    "id": "bundle--8f88a657-c532-42a4-9e9a-a183931001b9",
    "objects": [
        {
            "id": "windows-registry-key--16271437-052e-5f4f-b0f7-b1d5d79400dc",
            "spec_version": "2.1",
            "attribute_key": "HKEY_LOCAL_MACHINE\\System\\FooOnly\\BarOnly",
            "x_opencti_id": "8f03b8a9-52c4-4750-8537-934d57e1ec3b",
            "type": "windows-registry-key"
        },
        {
            "id": "windows-registry-key--70baa004-ee28-5b7f-b403-65118f4212ca",
            "spec_version": "2.1",
            "attribute_key": "hkey_local_machine\\system\\bar\\foo",
            "x_opencti_id": "c4ade1c4-aa6e-472d-a3f3-8082fed1b594",
            "type": "windows-registry-key"
        },
        {
            "id": "windows-registry-value-type--5b1777d9-aa42-508f-a6d5-0ce0d2da4fec",
            "spec_version": "2.1",
            "x_opencti_description": "hkey_local_machine\\\\system\\\\bar\\\\foo",
            "x_opencti_score": 50,
            "name": "Foo",
            "data": "qwerty",
            "data_type": "REG_SZ",
            "x_opencti_id": "9cadf7e6-da4d-4dee-8d11-319422b50368",
            "type": "windows-registry-value-type"
        }
    ]
}
$ stix2_validator --version 2.1 2021-04-10T18_50_55.421Z_TLP_ALL_\(ExportFileStix\)_Stix-Cyber-Observable_all.json
Patching requests.Session with class: type
================================================================================
[-] Results for: 2021-04-10T18_50_55.421Z_TLP_ALL_(ExportFileStix)_Stix-Cyber-Observable_all.json
[X] STIX JSON: Invalid
    [!] Warning: windows-registry-key--16271437-052e-5f4f-b0f7-b1d5d79400dc: {101} Cyber Observable Object custom property 'attribute_key' should start with 'x_' followed by a source unique identifier (like a domain name with dots replaced by hyphens), a hyphen and then the name.
    [!] Warning: windows-registry-key--70baa004-ee28-5b7f-b403-65118f4212ca: {101} Cyber Observable Object custom property 'attribute_key' should start with 'x_' followed by a source unique identifier (like a domain name with dots replaced by hyphens), a hyphen and then the name.
    [!] Warning: windows-registry-value-type--5b1777d9-aa42-508f-a6d5-0ce0d2da4fec: {101} Custom object type 'windows-registry-value-type' should start with 'x-' followed by a source unique identifier (like a domain name with dots replaced by hyphens), a hyphen and then the name.
    [!] Warning: windows-registry-value-type--5b1777d9-aa42-508f-a6d5-0ce0d2da4fec: {103} Given ID value windows-registry-value-type--5b1777d9-aa42-508f-a6d5-0ce0d2da4fec is not a valid UUIDv4 ID.
    [X] windows-registry-key--16271437-052e-5f4f-b0f7-b1d5d79400dc: {'id': 'windows-registry-key--16271437-052e-5f4f-b0f7-b1d5d79400dc', 'spec_version': '2.1', 'attribute_key': 'HKEY_LOCAL_MACHINE\\System\\FooOnly\\BarOnly', 'x_opencti_id': '8f03b8a9-52c4-4750-8537-934d57e1ec3b', 'type': 'windows-registry-key'} is not valid under any of the given schemas:
{'$id': 'http://raw.githubusercontent.com/oasis-open/cti-stix2-json-schemas/stix2.1/schemas/observables/windows-registry-key.json', '$schema': 'http://json-schema.org/draft-07/schema#', 'title': 'windows-registry-key', 'description': 'The Registry Key Object represents the properties of a Windows registry key.', 'type': 'object', 'allOf': [{'$ref': '../common/cyber-observable-core.json'}, {'properties': {'type': {'type': 'string', 'description': 'The value of this property MUST be `windows-registry-key`.', 'enum': ['windows-registry-key']}, 'id': {'title': 'id', 'pattern': '^windows-registry-key--'}, 'key': {'type': 'string', 'pattern': '^HKEY_LOCAL_MACHINE|hkey_local_machine|HKEY_CURRENT_USER|hkey_current_user|HKEY_CLASSES_ROOT|hkey_classes_root|HKEY_CURRENT_CONFIG|hkey_current_config|HKEY_PERFORMANCE_DATA|hkey_performance_data|HKEY_USERS|hkey_users|HKEY_DYN_DATA|hkey_dyn_data', 'description': 'Specifies the full registry key including the hive.'}, 'values': {'type': 'array', 'items': {'$ref': '#/definitions/windows-registry-value-type'}, 'description': 'Specifies the values found under the registry key.'}, 'modified_time': {'$ref': '../common/timestamp.json', 'description': 'Specifies the last date/time that the registry key was modified.'}, 'creator_user_ref': {'description': 'Specifies a reference to a user account, represented as a User Account Object, that created the registry key.', 'type': 'string'}, 'number_of_subkeys': {'type': 'integer', 'description': 'Specifies the number of subkeys contained under the registry key.'}}}], 'anyOf': [{'required': ['key']}, {'required': ['values']}, {'required': ['modified']}, {'required': ['creator_user_ref']}, {'required': ['number_of_subkeys']}], 'definitions': {'windows-registry-value-type': {'type': 'object', 'properties': {'name': {'type': 'string', 'description': 'Specifies the name of the registry value. For specifying the default value in a registry key, an empty string MUST be used.'}, 'data': {'type': 'string', 'description': 'Specifies the data contained in the registry value.'}, 'data_type': {'type': 'string', 'description': 'Specifies the registry (REG_*) data type used in the registry value.', 'enum': ['REG_NONE', 'REG_SZ', 'REG_EXPAND_SZ', 'REG_BINARY', 'REG_DWORD', 'REG_DWORD_BIG_ENDIAN', 'REG_DWORD_LITTLE_ENDIAN', 'REG_LINK', 'REG_MULTI_SZ', 'REG_RESOURCE_LIST', 'REG_FULL_RESOURCE_DESCRIPTION', 'REG_RESOURCE_REQUIREMENTS_LIST', 'REG_QWORD', 'REG_INVALID_TYPE']}}, 'anyOf': [{'required': ['name']}, {'required': ['data']}, {'required': ['data_type']}]}}}
    [X] windows-registry-key--70baa004-ee28-5b7f-b403-65118f4212ca: {'id': 'windows-registry-key--70baa004-ee28-5b7f-b403-65118f4212ca', 'spec_version': '2.1', 'attribute_key': 'hkey_local_machine\\system\\bar\\foo', 'x_opencti_id': 'c4ade1c4-aa6e-472d-a3f3-8082fed1b594', 'type': 'windows-registry-key'} is not valid under any of the given schemas:
{'$id': 'http://raw.githubusercontent.com/oasis-open/cti-stix2-json-schemas/stix2.1/schemas/observables/windows-registry-key.json', '$schema': 'http://json-schema.org/draft-07/schema#', 'title': 'windows-registry-key', 'description': 'The Registry Key Object represents the properties of a Windows registry key.', 'type': 'object', 'allOf': [{'$ref': '../common/cyber-observable-core.json'}, {'properties': {'type': {'type': 'string', 'description': 'The value of this property MUST be `windows-registry-key`.', 'enum': ['windows-registry-key']}, 'id': {'title': 'id', 'pattern': '^windows-registry-key--'}, 'key': {'type': 'string', 'pattern': '^HKEY_LOCAL_MACHINE|hkey_local_machine|HKEY_CURRENT_USER|hkey_current_user|HKEY_CLASSES_ROOT|hkey_classes_root|HKEY_CURRENT_CONFIG|hkey_current_config|HKEY_PERFORMANCE_DATA|hkey_performance_data|HKEY_USERS|hkey_users|HKEY_DYN_DATA|hkey_dyn_data', 'description': 'Specifies the full registry key including the hive.'}, 'values': {'type': 'array', 'items': {'$ref': '#/definitions/windows-registry-value-type'}, 'description': 'Specifies the values found under the registry key.'}, 'modified_time': {'$ref': '../common/timestamp.json', 'description': 'Specifies the last date/time that the registry key was modified.'}, 'creator_user_ref': {'description': 'Specifies a reference to a user account, represented as a User Account Object, that created the registry key.', 'type': 'string'}, 'number_of_subkeys': {'type': 'integer', 'description': 'Specifies the number of subkeys contained under the registry key.'}}}], 'anyOf': [{'required': ['key']}, {'required': ['values']}, {'required': ['modified']}, {'required': ['creator_user_ref']}, {'required': ['number_of_subkeys']}], 'definitions': {'windows-registry-value-type': {'type': 'object', 'properties': {'name': {'type': 'string', 'description': 'Specifies the name of the registry value. For specifying the default value in a registry key, an empty string MUST be used.'}, 'data': {'type': 'string', 'description': 'Specifies the data contained in the registry value.'}, 'data_type': {'type': 'string', 'description': 'Specifies the registry (REG_*) data type used in the registry value.', 'enum': ['REG_NONE', 'REG_SZ', 'REG_EXPAND_SZ', 'REG_BINARY', 'REG_DWORD', 'REG_DWORD_BIG_ENDIAN', 'REG_DWORD_LITTLE_ENDIAN', 'REG_LINK', 'REG_MULTI_SZ', 'REG_RESOURCE_LIST', 'REG_FULL_RESOURCE_DESCRIPTION', 'REG_RESOURCE_REQUIREMENTS_LIST', 'REG_QWORD', 'REG_INVALID_TYPE']}}, 'anyOf': [{'required': ['name']}, {'required': ['data']}, {'required': ['data_type']}]}}}
    [X] windows-registry-key--16271437-052e-5f4f-b0f7-b1d5d79400dc: If no Contributing Properties are present, a UUIDv4 must be used
    [X] windows-registry-key--70baa004-ee28-5b7f-b403-65118f4212ca: If no Contributing Properties are present, a UUIDv4 must be used
SamuelHassine commented 3 years ago

Linked to OpenCTI-Platform/client-python#155

misje commented 7 months ago

I found this issue after spending quite some time understanding why importing registry key values didn't work. I can confirm that this is still an issue in 6.0.7. However, there are no errors at all. values is just seemingly ignored.

I see a reference to a client-python issue, but that issue only seems to describe references to IDs(?). Importing objects like Network-Traffic with src_ref/dst_ref to IP address SCOs work. However, Windows-Registry-Key with values doesn't.