oasis-open / cti-python-stix2

OASIS TC Open Repository: Python APIs for STIX 2
https://stix2.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
362 stars 117 forks source link

TAXIICollectionSink collection fails the connection #247

Closed m0jtaba closed 5 years ago

m0jtaba commented 5 years ago

Used to be able to POST bundle via the method below but the last time I ran the config below was last year so something has changed but I cannot see what, any suggestions would be gratefully received.

I am able to hit and get response for: "http://127.0.0.1:5000/taxii/"

Traceback:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/stix2/datastore/taxii.py", line 67, in __init__
    if collection.can_write:
  File "/usr/lib/python3.6/site-packages/taxii2client/__init__.py", line 417, in can_write
    self._ensure_loaded()
  File "/usr/lib/python3.6/site-packages/taxii2client/__init__.py", line 480, in _ensure_loaded
    self.refresh()
  File "/usr/lib/python3.6/site-packages/taxii2client/__init__.py", line 495, in refresh
    headers={"Accept": accept})
  File "/usr/lib/python3.6/site-packages/taxii2client/__init__.py", line 941, in get
    resp.raise_for_status()
  File "/usr/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: NOT FOUND for url: http://127.0.0.1:5000/mitre/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "import_data.py", line 20, in <module>
    taxii_collection_sink = TAXIICollectionSink(collection, allow_custom=True)
  File "/usr/lib/python3.6/site-packages/stix2/datastore/taxii.py", line 78, in __init__
    " Collection object provided could not be reached. Receved error:", e,
stix2.datastore.DataSourceError: The underlying TAXII Collection resource defined in the supplied TAXII Collection object provided could not be reached. Receved error: "404 Client Error: NOT FOUND for url: http://127.0.0.1:5000/mitre/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116/"

Medallion:

medallion taxii_backend_mongo.json --host localhost --debug-mode 

taxii_backend_mongo.json:

{
    "backend": {
        "module": "medallion.backends.mongodb_backend",
        "module_class": "MongoBackend",
        "url": "mongodb://127.0.0.1:27017/"
    },
    "users": {
        "guest": "guest"
    }
}

Mongo Collection:

from generic_initialize_mongodb import (
    add_api_root, build_new_mongo_databases_and_collection, connect_to_client)

def main():
    """
    Main...
    """
    create_mitre_collection()

def create_mitre_collection():
    """
    creating the skeleton
    """
    client = connect_to_client()
    client.drop_database("discovery_database")
    mitre_database_collection = build_new_mongo_databases_and_collection(
        client)
    mitre_database_collection["discovery_information"].insert_one(
        {
            "title": "MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK™)",
            "description": "MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK™) is a curated knowledge base and model for cyber adversary behavior, reflecting the various phases of an adversary’s lifecycle and the platforms they are known to target. ATT&CK is useful for understanding security risk against known adversary behavior, for planning security improvements, and verifying defenses work as expected.",
            "contact": "foo",
            "api_roots": []})
    client.drop_database("mitre")
    api_root_db = add_api_root(
        client,
        url="http://127.0.0.1:5000/mitre/collections/",
        title="MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK™)",
        description="MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK™) is a curated knowledge base and model for cyber adversary behavior, reflecting the various phases of an adversary’s lifecycle and the platforms they are known to target. ATT&CK is useful for understanding security risk against known adversary behavior, for planning security improvements, and verifying defenses work as expected.",
        max_content_length=9765625,
        default=True)

    api_root_db["collections"].insert_one({
        "id": "91a7b528-80eb-42ed-a74d-c6fbd5a26116",
        "title": "ATT&CK for Enterprise",
        "description": "ATT&CK for Enterprise is an adversary behavior model that describes the actions an adversary may take while operating within an enterprise network.",
        "can_read": True,
        "can_write": True,
        "media_types": [
            "application/vnd.oasis.stix+json; version=2.0"
        ]})

    api_root_db["collections"].insert_one(
        {
            "id": "52892447-4d7e-4f70-b94d-d7f22742ff63",
            "title": "PRE-ATT&CK",
            "description": "PRE-ATT&CK provides the ability to prevent an attack before the adversary has a chance to get in. The seventeen tactic categories for PRE-ATT&CK were derived from the first four stages (recon, weaponize, deliver, and execute) of a seven-stage Cyber Attack Lifecycle2 (first articulated by Lockheed Martin as the Cyber Kill Chain®). This cyber threat framework captures the tactics, techniques, and procedures adversaries use to select a target, obtain information, and launch a campaign. The framework lists the ways that adversaries perform each tactic and provides the ability to track and organize adversary statistics and patterns. Ultimately, this arms defenders with a broader understanding of adversary actions that they can use to determine technical or policy-based mitigations and evaluate the quality and utility of cyber threat intelligence data sources.",
            "can_read": True,
            "can_write": True,
            "media_types": ["application/vnd.oasis.stix+json; version=2.0"]})

    api_root_db["collections"].insert_one(
        {
            "id": "c0c5fc01-4a76-4475-8df2-3ba34ad9e12b",
            "title": "Mobile Attack",
            "description": "The ATT&CK Mobile Profile builds upon NIST's Mobile Threat Catalogue, providing a model of adversarial tactics and techniques used to gain access to mobile devices as well as tactics and techniques to then take advantage of that access in order to accomplish adversarial objectives. The ATT&CK Mobile Profile also depicts network-based effects, which are adversarial tactics and techniques that an adversary can employ without access to the mobile device itself. Each adversarial technique includes a technical description along with applicable mitigation/countermeasure approaches, applicable detection analytics, and examples of use.",
            "can_read": True,
            "can_write": True,
            "media_types": ["application/vnd.oasis.stix+json; version=2.0"]})

if __name__ == "__main__":
    main()

import data:

import json
import socket
from time import sleep
import sys
from stix2 import TAXIICollectionSink
from taxii2client import Collection
#snippet
collection = Collection(
"http://127.0.0.1:5000/mitre/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116",
user="guest",
password="guest")

taxii_collection_sink = TAXIICollectionSink(collection, allow_custom=True)

data = json.load(open('enterprise-attack.json'))

taxii_collection_sink.add(data)

print("Attack added to TAXII Server")
m0jtaba commented 5 years ago

The issue has been resolved and for completeness the issue was with the mongodb creation: Before:

api_root_db = add_api_root(
        client,
        url="http://127.0.0.1:5000/mitre/collections/",
        title="MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK™)",
        description="MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK™) is a curated knowledge base and model for cyber adversary behavior, reflecting the various phases of an adversary’s lifecycle and the platforms they are known to target. ATT&CK is useful for understanding security risk against known adversary behavior, for planning security improvements, and verifying defenses work as expected.",
        max_content_length=9765625,
        default=True)

After:

api_root_db = add_api_root(
        client,
        url="http://127.0.0.1:5000/mitre/",
        title="MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK™)",
        description="MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK™) is a curated knowledge base and model for cyber adversary behavior, reflecting the various phases of an adversary’s lifecycle and the platforms they are known to target. ATT&CK is useful for understanding security risk against known adversary behavior, for planning security improvements, and verifying defenses work as expected.",
        max_content_length=9765625,
        default=True)

I was adding collection to the api root which is wrong.........