Elastic-ecs mapping - Email Object

[Harmedox]

Elastic Email Object is currently not mapped to STIX. As an example, an Email Object

"email": {
        "attachments": [
            {
            "file": {
                "name": "tabby.html",
                "mime_type": "text/html"
                }
            },
            {
            "file": {
                  "name": "tabby.zip",
                  "mime_type": "application/zip"
              }    
            }
        ],
        "subject": "Check out this picture of a cat!",
        "from": {
            "address": "from@address.com"
        },
        "to": {
            "address": [
                "to1@address.com",
                "to2@address.com"
            ]
        },
}

should be represented in the STIX bundle as:

{
    "0": {
        "type": "email-message",
        "subject": "Check out this picture of a cat!",
        "from_ref": "3",
        "to_refs": ["4", "5"],
        "is_multipart": true,
        "content_type": "multipart/mixed",
        "body-multiparts": [{
            "content_type": "text/html",
            "content_disposition": "attachment; filename=tabby.html",
            "body_raw_ref": "1"
        }, 
        {
            "content_type": "application/zip",
            "content_disposition": "attachment; filename=tabby.zip",
            "body_raw_ref": "2"
        }]
    },
    "1": {
        "type": "file",
        "name": "tabby.html",
        "mime_type": "text/html"
    },
    "2": {
        "type": "file",
        "name": "tabby.zip",
        "mime_type":  "application/zip"
    },
    "3": {
        "type": "email-addr",
        "value": "from@address.com"
    },
    "4": {
        "type": "email-addr",
        "value": "to1@address.com"
    },
    "5": {
        "type": "email-addr",
        "value": "to2@address.com"
    }
}

[Harmedox]

The significant challenges with this mapping are:

1. email.attachments is an array of objects.
2. The presence of an entity in the Email object leads to multiple attributes in STIX. For example, the presence of more than one object in the email.attachments array means that "is_multipart": true, "content_type": "multipart/mixed"...must be set in the STIX bundle.
3. For example, email.attachments.file.name and email.attachments.file.mime_type has to be transformed to form email-message.body_multiparts. content_type and email-message.body_multiparts. content_disposition

Anyone with ideas on how to handle (1) and (2) above? I fixed (3) using a transformer.

[Harmedox]

@mdazam1942 @delliott90 ^^^ any thoughts?

[mdazam1942]

In this case, the only way to set is_multipart and content_type is inside stix_shifter_modules/elastic_ecs/stix_transmission/connector.py as part of results processing.

there are few connectors that does the same. For example: https://github.com/opencybersecurityalliance/stix-shifter/blob/705881737ee698277a7fcb3245042a733c3065f8/stix_shifter_modules/gcp_chronicle/stix_transmission/results_connector.py#L381 https://github.com/opencybersecurityalliance/stix-shifter/blob/705881737ee698277a7fcb3245042a733c3065f8/stix_shifter_modules/proofpoint/stix_transmission/results_connector.py#L41

[subbyte]

@mdazam1942 just to confirm, does it mean patching the STIX bundle in the transmission module to add additional fields in the raw data in order for the translation module to pick them up and translate?

[mdazam1942]

Correct. add additional fields in the raw data and map those fields in to_stix. Results translator class should automatically pick them up while translating to stix observable.