search

muchdogesec / arango_taxii_server

A lightweight TAXII API wrapper for ArangoDB.
GNU Affero General Public License v3.0
2 stars 0 forks source link

`X-TAXII-Date-Added-Last` should show highest `modified` time of object on page #4

Open himynamesdave opened 3 weeks ago

himynamesdave commented 3 weeks ago 
{
  "more": true,
  "next": 35941940,
  "objects": [
    {
      "created": "2014-06-23T00:00:00.000Z",
      "created_by_ref": "identity--72e906ce-ca1b-5d73-adcd-9ea9eb66a1b4",
      "id": "relationship--b239b292-9552-5673-a07d-882fed673875",
      "modified": "2024-01-01T00:00:00.000Z",
      "object_marking_refs": [
        "marking-definition--17d82bb2-eeeb-4898-bda5-3ddbcd2b799d"
      ],
      "relationship_type": "created_by_ref",
      "source_ref": "attack-pattern--897a5506-45bb-4f6f-96e7-55f4c0b9021a",
      "spec_version": "2.1",
      "target_ref": "identity--e50ab59c-5c4f-4d40-bf6a-d58418d89bcd",
      "type": "relationship"
    },
    {
      "created": "2014-06-23T00:00:00.000Z",
      "created_by_ref": "identity--72e906ce-ca1b-5d73-adcd-9ea9eb66a1b4",
      "id": "relationship--ca5d9fe0-0a56-5a99-9e3b-bf0bb47aafda",
      "modified": "2024-01-01T00:00:00.000Z",
      "object_marking_refs": [
        "marking-definition--17d82bb2-eeeb-4898-bda5-3ddbcd2b799d"
      ],
      "relationship_type": "object_marking_refs",
      "source_ref": "attack-pattern--897a5506-45bb-4f6f-96e7-55f4c0b9021a",
      "spec_version": "2.1",
      "target_ref": "marking-definition--17d82bb2-eeeb-4898-bda5-3ddbcd2b799d",
      "type": "relationship"
    },
    {
      "created": "2014-06-23T00:00:00.000Z",
      "created_by_ref": "identity--e50ab59c-5c4f-4d40-bf6a-d58418d89bcd",
      "description": "In this attack pattern, the adversary monitors network traffic between nodes of a public or multicast network in an attempt to capture sensitive information at the protocol level. Network sniffing applications can reveal TCP/IP, DNS, Ethernet, and other low-level network communication information. The adversary takes a passive role in this attack pattern and simply observes and analyzes the traffic. The adversary may precipitate or indirectly influence the content of the observed transaction, but is never the intended recipient of the target information.",
      "external_references": [
        {
          "external_id": "CAPEC-158",
          "source_name": "capec",
          "url": "https://capec.mitre.org/data/definitions/158.html"
        },
        {
          "external_id": "CWE-311",
          "source_name": "cwe",
          "url": "http://cwe.mitre.org/data/definitions/311.html"
        },
        {
          "description": "Network Sniffing",
          "external_id": "T1040",
          "source_name": "ATTACK",
          "url": "https://attack.mitre.org/wiki/Technique/T1040"
        },
        {
          "description": "Multi-Factor Authentication Interception",
          "external_id": "T1111",
          "source_name": "ATTACK",
          "url": "https://attack.mitre.org/wiki/Technique/T1111"
        },
        {
          "description": "Acquire Access",
          "external_id": "T1650",
          "source_name": "ATTACK",
          "url": "https://attack.mitre.org/wiki/Technique/T1650"
        },
        {
          "description": "Hijack Execution Flow: ServicesFile Permissions Weakness",
          "external_id": "T1574.010",
          "source_name": "ATTACK",
          "url": "https://attack.mitre.org/wiki/Technique/T1574/010"
        }
      ],
      "id": "attack-pattern--897a5506-45bb-4f6f-96e7-55f4c0b9021a",
      "modified": "2024-01-01T00:00:00.000Z",
      "name": "UPDATE OBJECT 2ND TIME",
      "object_marking_refs": [
        "marking-definition--17d82bb2-eeeb-4898-bda5-3ddbcd2b799d"
      ],
      "spec_version": "2.1",
      "type": "attack-pattern",
      "x_capec_abstraction": "Detailed",
      "x_capec_can_follow_refs": [
        "attack-pattern--c9b31907-c466-4325-af55-c418aea8b964"
      ],
      "x_capec_child_of_refs": [
        "attack-pattern--bdcdc784-d891-4ca8-847b-38ddca37a6ec"
      ],
      "x_capec_consequences": {
        "Confidentiality": [
          "Read Data"
        ]
      },
      "x_capec_domains": [
        "Communications",
        "Software"
      ],
      "x_capec_prerequisites": [
        "The target must be communicating on a network protocol visible by a network sniffing application.",
        "The adversary must obtain a logical position on the network from intercepting target network traffic is possible. Depending on the network topology, traffic sniffing may be simple or challenging. If both the target sender and target recipient are members of a single subnet, the adversary must also be on that subnet in order to see their traffic communication."
      ],
      "x_capec_resources_required": [
        "A tool with the capability of presenting network communication traffic (e.g., Wireshark, tcpdump, Cain and Abel, etc.)."
      ],
      "x_capec_skills_required": {
        "Low": "Adversaries can obtain and set up open-source network sniffing tools easily."
      },
      "x_capec_status": "Draft",
      "x_capec_typical_severity": "Medium",
      "x_capec_version": "3.9"
    },
    {
      "created": "2014-06-23T00:00:00.000Z",
      "created_by_ref": "identity--72e906ce-ca1b-5d73-adcd-9ea9eb66a1b4",
      "id": "relationship--3730f73a-ec4a-5616-9545-7413e9c1f013",
      "modified": "2024-01-01T00:00:00.000Z",
      "object_marking_refs": [
        "marking-definition--17d82bb2-eeeb-4898-bda5-3ddbcd2b799d"
      ],
      "relationship_type": "x_capec_child_of_refs",
      "source_ref": "attack-pattern--897a5506-45bb-4f6f-96e7-55f4c0b9021a",
      "spec_version": "2.1",
      "target_ref": "attack-pattern--bdcdc784-d891-4ca8-847b-38ddca37a6ec",
      "type": "relationship"
    },
    {
      "created": "2014-06-23T00:00:00.000Z",
      "created_by_ref": "identity--72e906ce-ca1b-5d73-adcd-9ea9eb66a1b4",
      "id": "relationship--aa803d69-3572-537e-9699-69cc671a2f99",
      "modified": "2024-01-01T00:00:00.000Z",
      "object_marking_refs": [
        "marking-definition--17d82bb2-eeeb-4898-bda5-3ddbcd2b799d"
      ],
      "relationship_type": "x_capec_can_follow_refs",
      "source_ref": "attack-pattern--897a5506-45bb-4f6f-96e7-55f4c0b9021a",
      "spec_version": "2.1",
      "target_ref": "attack-pattern--c9b31907-c466-4325-af55-c418aea8b964",
      "type": "relationship"
    },
    {
      "created": "2017-01-02T00:00:00.000Z",
      "created_by_ref": "identity--72e906ce-ca1b-5d73-adcd-9ea9eb66a1b4",
      "id": "relationship--a1764b0d-3a2e-5456-981c-33afcdbe7d7d",
      "modified": "2024-01-01T00:00:00.000Z",
      "object_marking_refs": [
        "marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9",
        "marking-definition--762246cb-c8a1-53a7-94b3-eafe3ed511c9"
      ],
      "relationship_type": "object_marking_refs",
      "source_ref": "custom-sdo--cbc0b79a-ecbd-59f1-b45b-ea4730df1c2e",
      "spec_version": "2.1",
      "target_ref": "marking-definition--762246cb-c8a1-53a7-94b3-eafe3ed511c9",
      "type": "relationship"
    },
    {
      "created": "2017-01-02T00:00:00.000Z",
      "created_by_ref": "identity--72e906ce-ca1b-5d73-adcd-9ea9eb66a1b4",
      "id": "relationship--9b2d0351-de5a-56c4-b685-a72d75327690",
      "modified": "2024-01-01T00:00:00.000Z",
      "object_marking_refs": [
        "marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9",
        "marking-definition--762246cb-c8a1-53a7-94b3-eafe3ed511c9"
      ],
      "relationship_type": "created_by_ref",
      "source_ref": "custom-sdo--cbc0b79a-ecbd-59f1-b45b-ea4730df1c2e",
      "spec_version": "2.1",
      "target_ref": "identity--762246cb-c8a1-53a7-94b3-eafe3ed511c9",
      "type": "relationship"
    },
    {
      "created": "2017-01-02T00:00:00.000Z",
      "created_by_ref": "identity--72e906ce-ca1b-5d73-adcd-9ea9eb66a1b4",
      "id": "relationship--cfeaac56-d91d-579d-a315-1c22558c1324",
      "modified": "2024-01-01T00:00:00.000Z",
      "object_marking_refs": [
        "marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9",
        "marking-definition--762246cb-c8a1-53a7-94b3-eafe3ed511c9"
      ],
      "relationship_type": "object_marking_refs",
      "source_ref": "custom-sdo--cbc0b79a-ecbd-59f1-b45b-ea4730df1c2e",
      "spec_version": "2.1",
      "target_ref": "marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9",
      "type": "relationship"
    },
    {
      "created": "2017-01-20T00:00:00.000Z",
      "definition": {
        "tlp": "amber"
      },
      "definition_type": "tlp",
      "id": "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82",
      "name": "TLP:AMBER",
      "spec_version": "2.1",
      "type": "marking-definition"
    },
    {
      "created": "2017-01-20T00:00:00.000Z",
      "definition": {
        "tlp": "white"
      },
      "definition_type": "tlp",
      "id": "marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9",
      "name": "TLP:WHITE",
      "spec_version": "2.1",
      "type": "marking-definition"
    }
  ]
}

The header shows

access-control-allow-origin: * 
 allow: GET,POST,HEAD,OPTIONS 
 connection: close 
 content-length: 7107 
 content-type: application/taxii+json;version=2.1 
 cross-origin-opener-policy: same-origin 
 date: Wed,05 Jun 2024 10:39:51 GMT 
 referrer-policy: same-origin 
 server: gunicorn 
 vary: Accept,origin 
 x-content-type-options: nosniff 
 x-frame-options: DENY 
 x-taxii-date-added-first: 2014-06-23T00:00:00.000Z 
 x-taxii-date-added-last: 2017-01-20T00:00:00.000Z

x-taxii-date-added-first is correct. This should be the lowest created time of object in page

HOWEVER

x-taxii-date-added-last is incorrect. This is currently highest created time of object in page. It should show highest modified time of object on page.

himynamesdave commented 3 weeks ago

I see the change, and in the code looks like it should work, but

curl -X 'GET' \
  'http://127.0.0.1:8000/api/taxii2/cti_database/collections/mitre_attack_enterprise/objects/?limit=10&match%5Btype%5D=attack-pattern' \
  -H 'accept: application/json' \
  -H 'Authorization: Basic

Response header

access-control-allow-origin: * 
 allow: GET,POST,HEAD,OPTIONS 
 connection: close 
 content-length: 31516 
 content-type: application/taxii+json;version=2.1 
 cross-origin-opener-policy: same-origin 
 date: Thu,06 Jun 2024 12:10:35 GMT 
 referrer-policy: same-origin 
 server: gunicorn 
 vary: Accept,origin 
 x-content-type-options: nosniff 
 x-frame-options: DENY 
 x-taxii-date-added-first: 2014-06-23T00:00:00.000Z 
 x-taxii-date-added-last: 2020-09-17T18:25:33.796Z

but highest modified time in response is "modified": "2024-01-01T00:00:00.000Z"

Coincidentally I do see a modified time in the response that matches 2020-09-17T18:25:33.796

X-TAXII-Date-Added-Last.json

fqrious commented 3 weeks ago

should be fixed now

himynamesdave commented 2 weeks ago

now working correctly for

http://127.0.0.1:8000/api/taxii2/arango_taxii_server_tests_database/collections/mitre_attack_enterprise/objects

However i Have realised for

http://127.0.0.1:8000/api/taxii2/arango_taxii_server_tests_database/collections/mitre_attack_enterprise/manifest/

the response is slightly different

{
  "more": true,
  "next": "46833320_undef+0.30489662317927557",
  "objects": [
    {
      "date_added": "2017-01-20T00:00:00.000Z",
      "id": "marking-definition--34098fce-860f-48ae-8e50-ebd3cc5e41da",
      "media_type": "application/stix+json;version=2.1",
      "version": "2017-01-20T00:00:00.000Z"
    },
    {
      "date_added": "2017-01-20T00:00:00.000Z",
      "id": "marking-definition--5e57c739-391a-4eb3-b6be-7d15ca92d5ed",
      "media_type": "application/stix+json;version=2.1",
      "version": "2017-01-20T00:00:00.000Z"
    },

For this endpoint the x-taxii-date-added-first and x-taxii-date-added-last should be highest and lowest version value

fqrious commented 2 weeks ago

Fixed

himynamesdave commented 1 week ago

I noticed an issue with this approach, because some object don't have modified time

If no modified time in the object, then the _record_modified time should be used.

For all objects with a modified time, this value should always be used