forensic-architecture / mtriage

framework to orchestrate the download and analysis of media
Other
98 stars 16 forks source link

Add an mtbatch file #162

Closed breezykermo closed 3 years ago

breezykermo commented 3 years ago

As a solution for keeping information from the YAML that is used to run an mtriage pass (so that it can be displayed in mtriage-viewer), mtriage now reproduces the values in the config to an '.mtbatch' file that is produced. This is produced after the production of any set of elements (after every selector or analyser), and contain:

  1. An etype value, which is a string representing the etype. This is used by mtriage-viewer to infer which viewer is appropriate for the elements.
  2. A config value, which represents the value that was passed into the component (analyser or selector) that produced the elements. When a config has multiple analysis steps, the full config should be represented in each '.mtbatch' file. This is important so that reconstruction work doesn't need to be done: the '.mtbatch' file should contain all the information to understand what kind of elements are in the batch, and how they were produced.
  3. A stage value, which represents which specific component in the config was used to produce the elements that the '.mtbatch' refers to.

This is an example of the .mtbatch file produced by docs/tutorial/2/2a.yaml:

{
  "etype": "Audio",
  "config": {
    "folder": "media/demo_official/2",
    "select": {
      "name": "Local",
      "config": {
        "source": "data/demo/2audio"
      }
    },
    "analyse": {
      "name": "ConvertAudio",
      "config": {
        "output_ext": "mp3"
      }
    }
  },
  "stage": {
    "name": "ConvertAudio",
    "module": "analyser"
  }
}

I modified the test suite to accommodate the production of this file, and have added some basic tests for LocalStorage (there were none before), including a test that the meta file is written.