Depth Based Plots - SAA

[dennisgsmith]

Description

• Add plots to display depth domain data from Shape Accelerometer Array (SAA) sensors

  • Movement can be observed by comparing current reading with the initial readings. Views of the data will be by depth (or elevation) and show X, Y and plan views.
  • A time slider will show the data through time. 3D plots should be displayed.
  • Plots for reporting will show current readings along with selectable past surveys and baseline.

• SAA record the X,Y,Z coordinates and temperature of their sensors on a datalogger. MIDAS will coordinate with the site for additional processing requirements for the data.

  • Automated transmission of data through Campbell Scientific loggers
  • Extend Telemetry API for data upload (assumed HTTPPost)
    • Need en example CSJSON request payload/schema for XYZ data
  • Updates to allow user to add "SAA" equivalency table mappings to datalogger page in MIDAS (UI/core API)
  • Manual uploads of different file formats (.dux, .toa5, .dat, .rpp, .csv)
  • Need example files for each one of formats and try to upload using exiting uploader
  • Look at using equivalency table to allow uploads with recallable configuration
    • "Have something like the equivalency table/uploader parser where users can map columns/headers to different timeseries for Inclinometers, IPIs, SAAs"

Tasks

• [ ] Update Telemetry and Core API to allow manual and automated depth-based data upload (API)
  • Add equivalency table capabilities and static metadata that relates to the datalogger
• [ ] Add SAA datalogger page: equivalency table mappings (UI)
  • Static metadata properties need to be added for SAA type
• [ ] Add and SAA/Depth-Based plot types (UI)

Questions

• Are Inclinometers, IPIs sub-types of SAA sensors? How should they be categorized in MIDAS? As another Instrument type?
• Are there example files available for depth based plots in .dux, .toa5, .dat, .rpp. .csv?
• What does the data logger payload look like in CSJSON format?

[dennisgsmith]

Front end visualization overlaps with #126 item #3

[KevinJJackson]

Depth Based Plots working with current inclinometer implementation, time slider allows users to scroll through and see multiple traces per graph. 1 trace per set of measurements:

[KevinJJackson]

Workflow changes for SAA/IPI Dataloggers

Static Metadata for SAA/IPIs

1. Add option for user to select that the model is being used for SAA or IPI
   1. If true, show static metadata fields (send data alongside other datalogger data)
      1. Add new fields to data logger table or new static table (API)
   2. Prevent saving without all fields populated.

Field Mapping Table <-> Timeseries

1. During TOA5 Upload (after field mapping table exists)
   1. Create new dialog for selecting which fields belong to which instrument.
      1. Left column instrument selection
      2. Right column field selection
      3. Successive instrument selectors should not show already selected instruments
         1. Same for field selectors
      4. Once user is happy with their selections, clicking save will:
         1. Create a new time series (if there are no existing overlapping names) for each field in the selected instrument. (API)
         2. Map new timeseries ids to respective field mapping table (API)
         3. Grab units from file correlating to the selected field
         4. Possibly be able to do this from raw data preview, not just file upload

Data Logger Depth Based Plots (NEED FORMULA FOR DETERMINING A/B-DISPLACEMENT)

1. Select a configuration or:
2. Select the fields needed to populate the plots:
   1. Field X (a-increment ?)
   2. Field Y (b-increment ?)
   3. Field Z (depth ?)
   4. Temp
3. Once fields are selected generate plots with time slider. Similar to current batch plot implementation of depth plots.
4. Save selected fields into a configuration

[dennisgsmith]

A high-level logical continuation of the process described above:

SAA workflow example

1. Create depth based instrument (SAA Example [1])
2. Stub out "depth based timeseries" objects instrument type (SAA Example [2])
3. Upload and attempt to create regular timeseries objects from each field for a selected depth-based instrument
4. Equivalency table maps field names from data file to timeseries
   • Possibly automate the process of creating timeseries
   • If timeseries are generated at this step, field name-timeseries mappings can be too
     • We can also explore an easier "multi-select" UI
5. User (or automated process controlled by user) maps regular timeseries to "SAATimeseries"[2] (or other depth-based timeseries depending on instrument type) and saves configuration
6. Depth based timeseries are available to be queried and plotted!!!

[1] Depth Based Instrument Model Prototype (SAA)

type SAAInstrument struct {
    Instrument       // include all existing instrument properties
    NumSegments int  // used to generate number of sensors
    BottomElevation float32
}

[2] Depth Based Timeseries Model Prototype (SAA)

type SAASensor struct {
    ID uuid.UUID            // unique id
    InstrumentID uuid.UUID  // references parent instrument
    SegmentLength float32   // metadata property of sensor
    X uuid.UUID             // references timeseries
    Y uuid.UUID             // references timeseries
    Z uuid.UUID             // references timeseries
    Temp uuid.UUID          // references timeseries
    InitialX time.Time      // references timeseries_measurement (time)
    InitialY time.Time      // references timeseries_measurement (time)
    InitialZ time.Time      // references timeseries_measurement (time)
    InitialTemp time.Time   // references timeseries_measurement (time)
}

[dennisgsmith]

• API add instrument types to domain for SAA and IPI
• Add identifier to timeseries for instrument type
• Create SAA forms for sensors

1. User creates instrument (SAA) (instrument page ONLY)
2. User creates data logger that will be uploading SAA data (just like any other data logger)
3. User uploads the data and auto-creates (or manually creates) all timeseries for uploaded data
   • assumption: auto-timeseries-creation will be for ONE instrument
   • if timeseries with field name already exists, timeseries creation is ignored, but still gets mapped to field for equivalency table
   • Must not be mapped to a different instrument (this should not happen but should be enforced as db constraint)
4. User maps timeseries to "SAA Sensors" - new tab with stubbed out sensors depending on "num segments property of instrument"

[dennisgsmith]

Status: API complete, awaiting validation.