We need a way of recording web forum postings and credential dumps

Overview

CybOX needs to have a way to record computer-based 'evidence' of things bad guys are posting on web forums, IRC/ICQ chat logs, pastebin dumps and things like that. These items are useful for recording the history of what the bad guys are talking about, and would enable those tracking the bad guys to accurately record what they are talking about. This will help document and understand the intent, and will help provide STIX observation objects that contain 'proof' that Threat Intelligence Analysts can then use to create Threat Actor / Campaign / Indicator / TTP objects.

I believe these sorts of objects are needed to allow analysts to start properly using the higher order analysis objects within STIX.

A simple Web Page Text Object containing one or more 'snippets' would ensure that people could capture most of the content relevant to the webpage. I can even visualize Browser plugins to automatically create these objects when on other websites (select text, right-click, create CybOX Text Snippet Object)

After some slightly careful thought, I've got a couple of strawman objects for consideration. They are discussed below:

Recording a text snippet from a webpage

This object is designed to just simply grab some text from a webpage somewhere, and to record where it was recorded, when it was seen, how its was encoded and what the text actually was. Extra for experts would be to take a picture of the web page and to store it as a related Artifact Object.

{
    "type": "webpage-text-snippet",
    "id": "webpage-text-snippet--201f2d16-b0c8-40b2-92f2-27b636a9e244",
    "url": "https://badevilforum.org/bad/evenworse/likeDrEvil/posts/#3322145",
    "timestamp": "2015-03-01T13:00:00Z",
    "mime-type": "text/plain",
    "charset": "UTF-8",
    "snippets" : [
        {
            "line": 10,
            "text": "Im going to get your creditr card dump XYZBirdman"
        },
        {
            "line": 12,
            "text": "Like Im going to let you Dumbo"
        },
        {
            "line": 14,
            "text": "Hey lets hack Bigbank.com!"
        }

    ]
}

In this way an Analyst could record the conversations they were monitoring, and from that provide 'proof' that could back up their analysis. This could help people who as a group are targeting and monitoring particular threat actors.

Recording a complete webpage

This object is designed to record a complete webpage, to record where it was recorded, when it was seen, how its was encoded and how it was constructed.

{
    "type": "html-page",
    "id": "html-page--26ffb872-1dd9-446e-b6f5-d58527e5b5d2",
    "url": "http://cyboxproject.github.io/documentation/objects/",
    "mime-type": "text/html",
    "charset": "UTF-8",
    "size": 13042,
    "timestamp": "2015-03-01T13:00:00Z",
    "last-modified-timestamp": "2015-01-21T19:31:08Z",
    "description": "This webpage has a description that says this webpage has a description. How meta.",
    "cookies" : [
        {
            "name": "_gat_UA-2222222",
            "content": "1",
            "domain": ".mozilla.org",
            "path": "/",
            "size": 40,
            "expires": "2016-03-01T13:00:00Z",
            "http": true,
            "secure": false
        }
    ],
    "page-content" : "<html><title>....</body></html>",
}

Then you can have a relationship between the html page and Image File objects to show the media that are contained within them.

Or you can have a relationship between the HTML page and a HTTP Session object to show that the html page was accessed in the https session.

CybOXProject / schemas