search

OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
6.31k stars 932 forks source link

[PLAYBOOKS] The playbook does not create Observables based on Indicators #5313

Closed rattat0r closed 9 months ago

rattat0r commented 10 months ago

Description

I created a playbook for creating Observables based on Indicators. These Indicators are created by the RST Threat Feed external import connector. The playbook shows successful operation in the "Last execution traces" section, but new Observables are not created

Environment

  1. OS: Ubuntu 22.04
  2. OpenCTI version: OpenCTI 5.11.14
  3. OpenCTI client: frontend

Reproducible Steps

Steps to create the smallest reproducible scenario:

  1. Сreate a playbook listening for indicator creation image
  2. Create an indicator
  3. See that the playbook has worked, but the observable has not been created

Expected Output

Observable has been created

Actual Output

Observable has not been created

Additional information

Indicators are created by the RST Threat Feed connector

Screenshots

image image

SouadHadjiat commented 9 months ago

I reproduced the issue, here is the bundle we send for creation : 

[
  {
    "id": "38c497b4-d42a-4882-bc64-de4c33215005",
    "spec_version": "2.1",
    "type": "bundle",
    "objects": [
      {
        "id": "indicator--7ece00a2-ae39-5be0-bb9a-cbc784ef0a91",
        "spec_version": "2.1",
        "type": "indicator",
        "extensions": {
          "extension-definition--ea279b3e-5c71-4632-ac08-831c66a786ba": {
            "extension_type": "property-extension",
            "id": "e91d5d1b-f17a-4fe0-b38e-c05de2bafb99",
            "type": "Indicator",
            "created_at": "2023-12-29T11:08:15.637Z",
            "updated_at": "2023-12-29T11:08:15.637Z",
            "is_inferred": false,
            "creator_ids": [
              "a93d949b-b56d-4426-b7fe-b79ec3718b0e"
            ],
            "detection": false,
            "score": 10,
            "main_observable_type": "Email-Addr"
          }
        },
        "created": "2023-12-29T11:08:15.637Z",
        "modified": "2023-12-29T11:08:15.637Z",
        "revoked": false,
        "confidence": 75,
        "lang": "en",
        "name": "testbademailaddress2@email.com",
        "pattern": "[email-addr:value = 'testbademailaddress2@email.com']",
        "pattern_type": "stix",
        "valid_from": "2023-12-28T23:00:00.000Z",
        "valid_until": "2023-12-29T23:00:00.000Z"
      },
      {
        "type": "email-addr",
        "value": "testbademailaddress2@email.com",
        "extensions": {
          "extension-definition--f93e2c80-4231-4f9a-af8b-95c9bd566a82": {
            "description": "Simple observable of indicator {testbademailaddress2@email.com}"
          }
        }
      },
      {
        "id": "a14d0b26-e357-4352-95a0-3cddaedb0f7e",
        "type": "relationship",
        "source_ref": "indicator--7ece00a2-ae39-5be0-bb9a-cbc784ef0a91",
        "relationship_type": "based-on",
        "created": "2023-12-29T11:08:15.978Z",
        "modified": "2023-12-29T11:08:15.978Z"
      }
    ]
  }
]

the worker fails with an error because id is not present in the observable : 

{
  "timestamp": "2023-12-29T11:08:15.932490Z",
  "level": "ERROR",
  "name": "pycti.api",
  "message": "Traceback (most recent call last):\n  File \"/opt/opencti-worker/worker.py\", line 233, in data_handler\n    self.api.stix2.import_bundle_from_json(\n  File \"/usr/local/lib/python3.12/site-packages/pycti/utils/opencti_stix2.py\", line 216, in import_bundle_from_json\n    return self.import_bundle(\n           ^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/pycti/utils/opencti_stix2.py\", line 2332, in import_bundle\n    bundles = stix2_splitter.split_bundle(stix_bundle, False, event_version)\n              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/pycti/utils/opencti_stix2_splitter.py\", line 88, in split_bundle\n    raw_data[item[\"id\"]] = item\n             ~~~~^^^^^^\nKeyError: 'id'\n",
  "taskName": null
}
SouadHadjiat commented 9 months ago

Seems that "creating indicators based on observables" playbook step fails too. I get these errors (reproduced on testing) :

For indicator creation :

{
  "timestamp": "2024-01-02T09:41:56.246095Z",
  "level": "ERROR",
  "name": "api",
  "message": "[opencti_indicator] Missing parameters: name or pattern or pattern_type or x_opencti_main_observable_type",
  "exc_info": "NoneType: None",
  "taskName": null
}

For relationship creation :

{
  "timestamp": "2024-01-02T09:41:56.287122Z",
  "level": "ERROR",
  "name": "worker",
  "message": "{'name': 'Variable \"$input\" got invalid value \"f53f2d8f-b88f-4b29-8d7f-f1a13ee891c5\" at \"input.stix_id\"; Expected type \"StixId\". Provided value f53f2d8f-b88f-4b29-8d7f-f1a13ee891c5 is not a valid STIX ID', 'message': 'Variable \"$input\" got invalid value \"f53f2d8f-b88f-4b29-8d7f-f1a13ee891c5\" at \"input.stix_id\"; Expected type \"StixId\". Provided value f53f2d8f-b88f-4b29-8d7f-f1a13ee891c5 is not a valid STIX ID'}",
  "exc_info": "Traceback (most recent call last):\n  File \"/opt/opencti-worker/worker.py\", line 220, in data_handler\n    self.api.stix2.import_bundle_from_json(\n  File \"/usr/local/lib/python3.12/site-packages/pycti/utils/opencti_stix2.py\", line 215, in import_bundle_from_json\n    return self.import_bundle(\n           ^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/pycti/utils/opencti_stix2.py\", line 2353, in import_bundle\n    self.import_relationship(item, update, types)\n  File \"/usr/local/lib/python3.12/site-packages/pycti/utils/opencti_stix2.py\", line 1125, in import_relationship\n    stix_relation_result = self.opencti.stix_core_relationship.import_from_stix2(\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/pycti/entities/opencti_stix_core_relationship.py\", line 1141, in import_from_stix2\n    return self.create(\n           ^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/pycti/entities/opencti_stix_core_relationship.py\", line 618, in create\n    result = self.opencti.query(\n             ^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.12/site-packages/pycti/api/opencti_api_client.py\", line 344, in query\n    raise ValueError(\nValueError: {'name': 'Variable \"$input\" got invalid value \"f53f2d8f-b88f-4b29-8d7f-f1a13ee891c5\" at \"input.stix_id\"; Expected type \"StixId\". Provided value f53f2d8f-b88f-4b29-8d7f-f1a13ee891c5 is not a valid STIX ID', 'message': 'Variable \"$input\" got invalid value \"f53f2d8f-b88f-4b29-8d7f-f1a13ee891c5\" at \"input.stix_id\"; Expected type \"StixId\". Provided value f53f2d8f-b88f-4b29-8d7f-f1a13ee891c5 is not a valid STIX ID'}",
  "taskName": null
}

With this bundle : 

[
  {
    "id": "814e112e-a14e-407c-9440-39c39479424a",
    "spec_version": "2.1",
    "type": "bundle",
    "objects": [
      {
        "id": "email-addr--5278b3d1-64ff-5724-960c-161fca37956e",
        "spec_version": "2.1",
        "type": "email-addr",
        "extensions": {
          "extension-definition--ea279b3e-5c71-4632-ac08-831c66a786ba": {
            "extension_type": "property-extension",
            "id": "0f48ffe5-78d2-47f4-9bde-32dfef9d04cc",
            "type": "Email-Addr",
            "created_at": "2024-01-02T09:41:55.191Z",
            "updated_at": "2024-01-02T09:41:55.191Z",
            "is_inferred": false,
            "creator_ids": [
              "a93d949b-b56d-4426-b7fe-b79ec3718b0e"
            ]
          },
          "extension-definition--f93e2c80-4231-4f9a-af8b-95c9bd566a82": {
            "extension_type": "property-extension",
            "score": 10
          }
        },
        "value": "test-bad1@opencti.io"
      },
      {
        "id": "indicator--e65882c1-5727-5c67-a466-bb72ca294436",
        "spec_version": "2.1",
        "type": "indicator",
        "extensions": {
          "extension-definition--ea279b3e-5c71-4632-ac08-831c66a786ba": {
            "extension_type": "property-extension",
            "id": "ff763ae2-a47e-461b-b192-0c37a5e30008",
            "type": "Indicator"
          }
        },
        "name": "test-bad1@opencti.io",
        "pattern": "[email-addr:value = 'test-bad1@opencti.io']"
      },
      {
        "id": "f53f2d8f-b88f-4b29-8d7f-f1a13ee891c5",
        "type": "relationship",
        "source_ref": "indicator--e65882c1-5727-5c67-a466-bb72ca294436",
        "target_ref": "email-addr--5278b3d1-64ff-5724-960c-161fca37956e",
        "relationship_type": "based-on",
        "created": "2024-01-02T09:41:55.324Z",
        "modified": "2024-01-02T09:41:55.324Z"
      }
    ]
  }
]