This repository contains the draft code used to explore and analyze the data in the 12/2020 "Technical Scenario" document for VAULT. It is organized into a set of Jupyter notebooks runnable on any Linux or Mac system. For notebooks without interactive plots, the notebook is provided with output embedded directly into it, so that the results can be seen without having to set up and execute the code. Notebooks without output included are meant to be viewed "live", with a running Python server, so that the data can be fully explored interactively. PDF copies of all notebooks are provided for quick skimming or in case the notebook code or data is not available for running. Where appropriate, you can also visit a deployed version of the code.
To understand our algorithm and approach, please see our write-up at High Performance Hit Finder.
To get started with this codebase, see the Quickstart.
You can access deployed versions of the notebooks and dashboard at http://bit.ly/attvault, though these will be taken down at some point after the demo presentation.
See Downloading Data
The notebooks fall into the following categories:
These notebooks start with raw data where possible, with a goal of revealing it as it is, with as little cleanup as possible, so that same process can be applied to new data. These are primarily self contained, not relying on external scripts or modules in this repository (just packages in the Python environment installed).
These notebooks also focus on data, but on derived or computed values.
These files start with processed/prepared data, and approximate an end-user task (e.g. hit detection).
These files start with raw data and create cleaned/consolidated/computed data for use in the other categories. Many of these rely on scripts in scripts/
, where you can see the detailed computations involved.
These are all in the scripts/
subdirectory. Most print useful help with given the --help
option, or in their file docstrings.
hit_finder.py
: CLI tool for computing all vessels & times visible to a satellitereverse-hit-finder.py
: CLI tool for computing all satellites visible to a particular vessel MMSIintersect.py
: The core logic of the visibility intersection algorithminterpolate_ais.py
: Generates HDF5 files with synthetic interpolated points for vessel motionbuild_index_parallel.sh
: Parallel driver for sathelpers.py
, to precompute satellite trajectories.This page serves as the main instruction index. From here, you can navigate to various resources, deliverables, and documention specific to that process.
DoD/government documents:
Data files:
General background: