search

oasis-open / cti-stix-elevator

OASIS Cyber Threat Intelligence (CTI) TC Open Repository: Convert STIX 1.2 XML to STIX 2.x JSON
https://stix2-elevator.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
49 stars 23 forks source link

stix_stepper outputs duplicate SCOs #286

Open pcoccoli opened 2 years ago

pcoccoli commented 2 years ago

When a SCO appears multiple times in a STIX 2.0 bundle, the stix_stepper 2.1 output includes duplicate copies of that SCO.

Input file (conns2.json):

{
  "type": "bundle",
  "spec_version": "2.0",
  "id": "bundle--60abd6c0-c1eb-4f60-b687-12d8fd9a12cf",
  "objects": [
    {
      "type": "identity",
      "id": "identity--aef0e832-21ad-48b1-b7f4-49922ee97008",
      "created": "2021-03-01T20:13:56.957Z",
      "modified": "2021-03-01T20:13:56.957Z",
      "name": "stix-generator.py",
      "identity_class": "program"
    },
    {
      "type": "observed-data",
      "id": "observed-data--1f8605b8-84d8-4cf6-a71f-f3d52b67844d",
      "created_by_ref": "identity--aef0e832-21ad-48b1-b7f4-49922ee97008",
      "created": "2021-03-01T20:14:02.628Z",
      "modified": "2021-03-01T20:14:02.628Z",
      "first_observed": "2021-02-22T00:00:00.833739Z",
      "last_observed": "2021-02-22T00:04:59.06578Z",
      "number_observed": 1,
      "objects": {
        "0": {
          "type": "ipv4-addr",
          "value": "192.168.100.138"
        },
        "1": {
          "type": "ipv4-addr",
          "value": "192.168.1.152"
        },
        "2": {
          "type": "network-traffic",
          "start": "2021-02-22T00:00:00.833739Z",
          "end": "2021-02-22T00:04:59.06578Z",
          "src_ref": "0",
          "dst_ref": "1",
          "src_port": 52253,
          "dst_port": 22,
          "protocols": [
            "ip",
            "tcp"
          ],
          "src_byte_count": 602468,
          "dst_byte_count": 2709407
        }
      }
    },
    {
      "type": "observed-data",
      "id": "observed-data--99c6d673-297e-410d-ba05-14c9831743ce",
      "created_by_ref": "identity--aef0e832-21ad-48b1-b7f4-49922ee97008",
      "created": "2021-03-01T20:14:19.309Z",
      "modified": "2021-03-01T20:14:19.309Z",
      "first_observed": "2021-02-22T00:03:00.326094Z",
      "last_observed": "2021-02-22T00:04:41.438821Z",
      "number_observed": 1,
      "objects": {
        "0": {
          "type": "ipv4-addr",
          "value": "192.168.100.138"
        },
        "1": {
          "type": "ipv4-addr",
          "value": "192.168.1.231"
        },
        "2": {
          "type": "network-traffic",
          "start": "2021-02-22T00:03:00.326094Z",
          "end": "2021-02-22T00:04:41.438821Z",
          "src_ref": "0",
          "dst_ref": "1",
          "src_port": 60075,
          "dst_port": 8080,
          "protocols": [
            "ip",
            "tcp",
            "http"
          ],
          "src_byte_count": 43833,
          "dst_byte_count": 1021483
        }
      }
    }
  ]
}

Output:

$ stix_stepper conns2.json 
[stix2elevator.options] [214] [INFO   ] [2022-08-04 07:47:02,066] incidents option was not given, but it defaults to true for version 2.1
[stix2elevator.options] [214] [INFO   ] [2022-08-04 07:47:02,067] infrastructure option was not given, but it defaults to true for version 2.1
{
    "id": "bundle--60abd6c0-c1eb-4f60-b687-12d8fd9a12cf",
    "objects": [
        {
            "created": "2021-03-01T20:13:56.957Z",
            "id": "identity--aef0e832-21ad-48b1-b7f4-49922ee97008",
            "identity_class": "program",
            "modified": "2021-03-01T20:13:56.957Z",
            "name": "stix-generator.py",
            "spec_version": "2.1",
            "type": "identity"
        },
        {
            "id": "ipv4-addr--0ef668d6-7395-5db6-89ac-63e09a208289",
            "type": "ipv4-addr",
            "value": "192.168.100.138"
        },
        {
            "id": "ipv4-addr--39550498-7631-5c21-b92f-5bdcccb5868f",
            "type": "ipv4-addr",
            "value": "192.168.1.152"
        },
        {
            "dst_byte_count": 2709407,
            "dst_port": 22,
            "dst_ref": "ipv4-addr--39550498-7631-5c21-b92f-5bdcccb5868f",
            "end": "2021-02-22T00:04:59.06578Z",
            "id": "network-traffic--51cbba82-cd37-50c9-89b4-2a8fd3d4295f",
            "protocols": [
                "ip",
                "tcp"
            ],
            "src_byte_count": 602468,
            "src_port": 52253,
            "src_ref": "ipv4-addr--0ef668d6-7395-5db6-89ac-63e09a208289",
            "start": "2021-02-22T00:00:00.833739Z",
            "type": "network-traffic"
        },
        {
            "created": "2021-03-01T20:14:02.628Z",
            "created_by_ref": "identity--aef0e832-21ad-48b1-b7f4-49922ee97008",
            "first_observed": "2021-02-22T00:00:00.833739Z",
            "id": "observed-data--1f8605b8-84d8-4cf6-a71f-f3d52b67844d",
            "last_observed": "2021-02-22T00:04:59.06578Z",
            "modified": "2021-03-01T20:14:02.628Z",
            "number_observed": 1,
            "object_refs": [
                "ipv4-addr--0ef668d6-7395-5db6-89ac-63e09a208289",
                "ipv4-addr--39550498-7631-5c21-b92f-5bdcccb5868f",
                "network-traffic--51cbba82-cd37-50c9-89b4-2a8fd3d4295f"
            ],
            "spec_version": "2.1",
            "type": "observed-data"
        },
        {
            "id": "ipv4-addr--0ef668d6-7395-5db6-89ac-63e09a208289",
            "type": "ipv4-addr",
            "value": "192.168.100.138"
        },
        {
            "id": "ipv4-addr--32c52c41-6b3b-5e8f-a290-5c96a7960595",
            "type": "ipv4-addr",
            "value": "192.168.1.231"
        },
        {
            "dst_byte_count": 1021483,
            "dst_port": 8080,
            "dst_ref": "ipv4-addr--32c52c41-6b3b-5e8f-a290-5c96a7960595",
            "end": "2021-02-22T00:04:41.438821Z",
            "id": "network-traffic--ef5f0d9f-5296-5dcd-b897-6db2f0e49b57",
            "protocols": [
                "ip",
                "tcp",
                "http"
            ],
            "src_byte_count": 43833,
            "src_port": 60075,
            "src_ref": "ipv4-addr--0ef668d6-7395-5db6-89ac-63e09a208289",
            "start": "2021-02-22T00:03:00.326094Z",
            "type": "network-traffic"
        },
        {
            "created": "2021-03-01T20:14:19.309Z",
            "created_by_ref": "identity--aef0e832-21ad-48b1-b7f4-49922ee97008",
            "first_observed": "2021-02-22T00:03:00.326094Z",
            "id": "observed-data--99c6d673-297e-410d-ba05-14c9831743ce",
            "last_observed": "2021-02-22T00:04:41.438821Z",
            "modified": "2021-03-01T20:14:19.309Z",
            "number_observed": 1,
            "object_refs": [
                "ipv4-addr--0ef668d6-7395-5db6-89ac-63e09a208289",
                "ipv4-addr--32c52c41-6b3b-5e8f-a290-5c96a7960595",
                "network-traffic--ef5f0d9f-5296-5dcd-b897-6db2f0e49b57"
            ],
            "spec_version": "2.1",
            "type": "observed-data"
        }
    ],
    "type": "bundle"
}

In the output, the SCO ipv4-addr--0ef668d6-7395-5db6-89ac-63e09a208289 appears twice.